-- Logs begin at Sat 2022-09-03 00:43:59 UTC, end at Sat 2022-09-03 01:53:28 UTC. -- Sep 03 01:13:08.574553 dv-cinder-838009-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 03 01:13:08.585438 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 3 01:13:08 2022] *** Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: nodename: dv-cinder-838009-7-iscsi Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: machine: x86_64 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: clock source: unix Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: pcre jit disabled Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: detected number of CPU cores: 4 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: current working directory: / Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: detected binary path: /usr/bin/uwsgi-core Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: your processes number limit is 47680 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: your memory page size is 4096 bytes Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: detected max file descriptor number: 2048 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: lock engine: pthread robust mutexes Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: thunder lock: enabled Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 03 01:13:08.602097 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 03 01:13:08.626616 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: Python main interpreter initialized at 0x559c62ad59c0 Sep 03 01:13:08.626616 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: python threads support enabled Sep 03 01:13:08.626616 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: your server socket listen backlog is limited to 100 connections Sep 03 01:13:08.626616 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: your mercy for graceful operations on workers is 90 seconds Sep 03 01:13:08.626944 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: mapped 403077 bytes (393 KB) for 2 cores Sep 03 01:13:08.627051 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: *** Operational MODE: preforking *** Sep 03 01:13:08.627326 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: *** uWSGI is running in multiple interpreter mode *** Sep 03 01:13:08.627326 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: spawned uWSGI master process (pid: 81615) Sep 03 01:13:08.627535 dv-cinder-838009-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 03 01:13:08.629954 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: spawned uWSGI worker 1 (pid: 81627, cores: 1) Sep 03 01:13:08.629954 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: spawned uWSGI worker 2 (pid: 81628, cores: 1) Sep 03 01:13:08.629954 dv-cinder-838009-7-iscsi devstack@keystone.service[81615]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 03 01:13:13.001433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 01:13:13.001433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: from cryptography.utils import int_from_bytes Sep 03 01:13:13.002202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 01:13:13.002202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: from cryptography.utils import int_from_bytes Sep 03 01:13:13.002324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 01:13:13.002324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: from cryptography.utils import int_from_bytes Sep 03 01:13:13.002784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 03 01:13:13.002784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: from cryptography.utils import int_from_bytes Sep 03 01:13:13.052376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 03 01:13:13.052528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 03 01:13:13.052662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 03 01:13:13.052802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] command line args: [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 03 01:13:13.052890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 03 01:13:13.052972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 03 01:13:13.053051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] command line args: [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 03 01:13:13.053134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ================================================================================ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 03 01:13:13.053310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 03 01:13:13.053407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] admin_token = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.053485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ================================================================================ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 03 01:13:13.053562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config_dir = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.053739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] admin_token = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.053831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.053903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config_source = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config_dir = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] debug = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config_source = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] debug = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.054911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_config_append = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.055960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_dir = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_config_append = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_dir = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.056859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.057892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.058996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.059831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] publish_errors = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] publish_errors = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.060883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.061009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.061207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.061794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.061910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.061988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] standard_threads = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] standard_threads = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.062972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_journal = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_journal = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_json = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_json = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_stderr = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_syslog = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_stderr = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.063982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_syslog = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.064070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.064219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 03 01:13:13.064296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.064483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.064561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.064731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.064888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.064988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065072 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.065991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.066998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.067978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.068994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.069974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.070941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.071979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.072938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.073949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.074949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.075949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.076941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.077907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.078026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.078196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.078456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.078558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085260 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.085930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.086946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.087915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.088945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089519 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.089961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.090932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.092092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.092164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.092768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.092841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.092919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.092995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.external = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.password = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.093986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.token = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.external = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.password = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.094960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.token = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.095979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.096947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.097969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.098946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.099921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.100962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.101923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.102890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.103851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.104990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.105931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.106995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.107977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.108562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.108777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.108933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.109930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.110010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.110174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.110263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.110444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.110524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.113867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.114967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.115941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.116975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.117988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118652 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.118959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.119876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.120896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.121992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.122917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.123951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.124076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.124207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.124332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.124456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.124582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.124955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.125877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.126908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.127921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.128936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.129953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.130996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.131120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.131253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.131379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.131503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.131627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.131832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.132995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.133995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.134877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.135910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.136918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.137042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.137166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.137306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.137306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.137542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.137879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.138918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.139948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.140953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.141093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.141217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.141339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.141464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.141592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.141868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.142955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 03 01:13:13.143813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 03 01:13:13.171573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.171798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.172022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.173277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.173517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.173751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.180433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.182285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.216509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.216657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.216744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.216823 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.274709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.275461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.327602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.327850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.328574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.328815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.484929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.485161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.485388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.485474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.485610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.485818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.485905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.486039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.486191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.486512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.486735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.489688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.510935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.511865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.512015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.512109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.512271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.512353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.512510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.512731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.523788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.527967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.540990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.541074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.541208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.541294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.541425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.541510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.541641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 03 01:13:13.558881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.561817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 01:13:13.561915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.561915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 01:13:13.562117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 01:13:13.562321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 03 01:13:13.562575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.562952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.570852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.571139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.571373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.571481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.571600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.571703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.571825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.572110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.579887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.579987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.581643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.581934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.582077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.582224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.582384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.582543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.582673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.582835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.582945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.583937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.584052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.584217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.584327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.584491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.584604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.584899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.607169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.608592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.609473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.609597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.609767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.609904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.614500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.614750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.614835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.615041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.615163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.619260 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.621451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.621748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.621933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.622226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.628341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.628640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.628933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.629018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.629227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.629313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.629515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.629598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.629940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.630961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.646162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.646481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.646884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.647166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.650373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.651057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.653617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.653921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.654274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.654598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.658221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.658982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.661392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.662095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.664631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.664933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.665226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.665340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.665528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.665650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.665811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.665936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.666094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.666229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.666538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.666851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.682177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.682496 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.682781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.683056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.683289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.683572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.683847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.684113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.690499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.691703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.692021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.693092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.693537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.693821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.694650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.694943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.696547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.696838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.697144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.697434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.697646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.697852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.697937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.698206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.698523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.698904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.709212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.709503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.709794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.710076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.710183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.710384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.710485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.710675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.710760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.710962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.711046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.711350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.711636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.711922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.726848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.727315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.729768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.730200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.732692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.733621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.734817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.735291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.736315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.736607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.736765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.737018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.737103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.737436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.743450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.743907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.744928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.745358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.747863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.748266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.748352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.748669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.748757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.749040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.749126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.749322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.756904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.757162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.757264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.757478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.757561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.757774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.764966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.765252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.765340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 03 01:13:13.765571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.765666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.765823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.765909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.766950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.767870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.768848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 03 01:13:13.803492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559c62ad59c0 pid: 81627 (default app) Sep 03 01:13:13.803705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559c62ad59c0 pid: 81628 (default app) Sep 03 01:13:13.806887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3b54-8040-436f-a812-51457f6342c5 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:13.807182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3b54-8040-436f-a812-51457f6342c5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:13.807458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f3b54-8040-436f-a812-51457f6342c5 None None] PATH_INFO: `/v3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:13.812219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Sat Sep 3 01:13:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:21.276892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:21.277385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:21.277464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:21.283261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:13:21.289049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:21.289534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:21.290005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:21.369342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: INFO dbcounter [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] Registered counter for database keystone Sep 03 01:13:21.390292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG oslo_db.sqlalchemy.engines [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81627) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 03 01:13:21.396817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writer thread running {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 03 01:13:21.575021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG passlib.handlers.bcrypt [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81627) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 03 01:13:21.575328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG passlib.handlers.bcrypt [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81627) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 03 01:13:21.633798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:21.635423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:21.767961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 03 01:13:22.206291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:22.276737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: INFO dbcounter [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] Registered counter for database keystone Sep 03 01:13:22.300124 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG oslo_db.sqlalchemy.engines [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 03 01:13:22.306788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writer thread running {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 03 01:13:22.428321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50843d19-fa8c-4cf6-8218-fbb8c1f0fc7e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:22.432752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:22.433382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:22.433667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:22.433937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:22.442975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:22.494513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 03 01:13:22.494807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fdd8c-52d3-49d0-b5df-f684361388a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:22.505164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Sat Sep 3 01:13:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:13:22.511347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:22.526687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6960f-096e-4b83-9cde-9d50bdc8ebcc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:22.531084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:22.532187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:22.532464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:22.532740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] PATH_INFO: `/v3/projects/06b321cae6a24e3a9217594a16dab365` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:22.534291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] RBAC: Authorizing `identity:get_project(project_id=06b321cae6a24e3a9217594a16dab365)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:22.586993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING keystone.common.rbac_enforcer.enforcer [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 03 01:13:22.587357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d24c26-fe78-468a-bf58-2dad7eae0933 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:22.591965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Sep 3 01:13:22 2022] GET /identity/v3/projects/06b321cae6a24e3a9217594a16dab365 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:24.470682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a279f6-004c-4a89-9a9a-d85e270dcef9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:24.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a279f6-004c-4a89-9a9a-d85e270dcef9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:24.471281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a279f6-004c-4a89-9a9a-d85e270dcef9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:24.475319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:24.480543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:24.480967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:24.481095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:24.501500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:24.503043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:24.510736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 03 01:13:24.776526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a279f6-004c-4a89-9a9a-d85e270dcef9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:24.789153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a279f6-004c-4a89-9a9a-d85e270dcef9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:24.793323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e710fd6-8a9d-4931-9994-fe979558b9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:24.793922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710fd6-8a9d-4931-9994-fe979558b9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:24.794202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710fd6-8a9d-4931-9994-fe979558b9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:24.794511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e710fd6-8a9d-4931-9994-fe979558b9a1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:24.802562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e710fd6-8a9d-4931-9994-fe979558b9a1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:24.804054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e710fd6-8a9d-4931-9994-fe979558b9a1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:24.812829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Sat Sep 3 01:13:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:13:24.818218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:24.827826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689b8c1-ae65-4ad7-b1b2-16cccc70c2b2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:24.832146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d74d2e-ad90-4df5-b2a2-b6946d3bae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:24.832769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74d2e-ad90-4df5-b2a2-b6946d3bae9f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:24.833051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74d2e-ad90-4df5-b2a2-b6946d3bae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:24.833322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74d2e-ad90-4df5-b2a2-b6946d3bae9f admin admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:24.834640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d74d2e-ad90-4df5-b2a2-b6946d3bae9f admin admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:24.836486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d74d2e-ad90-4df5-b2a2-b6946d3bae9f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:24.837980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Sat Sep 3 01:13:24 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:26.803963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff4d9a-055b-4038-9262-18e16e006221 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:26.804451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff4d9a-055b-4038-9262-18e16e006221 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:26.804544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff4d9a-055b-4038-9262-18e16e006221 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:26.805865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:26.811469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:26.811902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:26.812142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:26.830867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:26.832053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:26.873562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6083a-a0d3-4bd9-910f-7792e6f14b57 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:26.873939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6083a-a0d3-4bd9-910f-7792e6f14b57 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:26.874418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6083a-a0d3-4bd9-910f-7792e6f14b57 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:26.875756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:13:26.880864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:26.881150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:26.881426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:26.914573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:26.930509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG passlib.handlers.bcrypt [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81628) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 03 01:13:26.931170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG passlib.handlers.bcrypt [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81628) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 03 01:13:26.981380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:26.983667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:27.083415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:27.183345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.208208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.299036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fd1b4-3498-4787-876a-5ee9336ed630 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.336754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.337310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.337589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.337858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.349660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:27.358831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:27.359927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:27.361514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:27.362937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.365400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.444693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.461317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932901-0f5f-4b1c-b588-257816d43727 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.462514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0349c-6172-46d1-bd99-75a9f84a3313 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.463687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.497232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.498366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.498679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.498948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.499615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:27.499917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:27.500940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.501376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:27.501852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.502411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.502736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.513110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:27.518792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.570677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:27.572239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:27.621914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5386498-67df-413e-b9fa-b12000ecb331 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.656241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.656878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.657161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.657436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.658587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:27.660062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:27.661639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:27.667292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.672615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 8/17] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:13:27.677894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.680608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.682096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.766397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eb0ca-39a0-41a6-90d0-536d802fabda None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.774901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639fac4-6e14-40d7-a6ab-1e8e7e079dd5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.799436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.800079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.800366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.800643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.801779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:27.803294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:27.804880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:27.811110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.812468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.813325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.813606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.813876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.832273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:27.833740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:13:27.851747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:27.852666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:13:27.853477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:13:27.853477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:13:27.853477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:27.853477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:13:27.859153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:27.899843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.916949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:27.919152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17299eff-26c7-48a5-a47d-89a6aaebc7c2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.953077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.953844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.954132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.954440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:27.955175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.domain.id=default, user.id=3188ed8aefba4edeb5f28b0b4a901aed) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:27.955481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:27.956948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92079441-9728-4d59-b7ae-bf01a4b1cbfc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:27.962307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d7974-4234-49df-8f7a-0e7340557353 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:27.963476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/role_assignments?user.id=3188ed8aefba4edeb5f28b0b4a901aed&scope.domain.id=default&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:27.998247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:27.998916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:27.999196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:27.999462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:28.000128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:28.000431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:28.001834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb4141-691e-4b21-9f51-a36fffcd2af9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:28.021927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:30.054037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0438f6-d057-4de9-a8da-dfb5f682bf2c None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.055049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0438f6-d057-4de9-a8da-dfb5f682bf2c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.055544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0438f6-d057-4de9-a8da-dfb5f682bf2c None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.057113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:30.062904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c6e7-48de-4e26-b5dc-fe23dd5da9af None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.063710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c6e7-48de-4e26-b5dc-fe23dd5da9af None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.064190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c6e7-48de-4e26-b5dc-fe23dd5da9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.091356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4868c6e7-48de-4e26-b5dc-fe23dd5da9af None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:30.092968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4868c6e7-48de-4e26-b5dc-fe23dd5da9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:30.172471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:30.302372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dd47e-887a-45e7-ace6-17801db72abc None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.302993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dd47e-887a-45e7-ace6-17801db72abc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.303485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dd47e-887a-45e7-ace6-17801db72abc None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.305118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:30.320339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.320958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.321432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.351360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:30.352911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:30.378071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eeec3-573a-4a15-901c-d4c9a9784085 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.378702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eeec3-573a-4a15-901c-d4c9a9784085 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.379172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eeec3-573a-4a15-901c-d4c9a9784085 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.380691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 14/25] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:13:30.385864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.386156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.386472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.406220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:30.407620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:30.420991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:30.471249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 12/26] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:30.514985 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 15/27] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:30.571429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:30.691088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99eb5d-4cf2-471d-8cb4-61fde7f330d9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:30.739604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:30.740251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:30.740532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:30.740806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:30.753816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:30.760507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:30.761378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:30.762141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 13/28] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:30.769077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:30.819338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:30.909757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:30.928817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:30.986125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b8fd1-24f6-42b1-91c7-36481af45e02 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.071946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.072562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.072842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.073115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.073794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:31.074189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.078635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.082320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f5e49-41ba-4439-9fc2-edbdf7fd4d55 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.096205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 16/29] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:31.101754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:31.166827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.167369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.167648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.167921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.169451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.173886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.241167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a3d4c-19dd-4b43-9d30-798d0351217d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.277861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a2e5d-3618-4540-88e9-ffaeae12d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.278527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2e5d-3618-4540-88e9-ffaeae12d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.278806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2e5d-3618-4540-88e9-ffaeae12d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.279672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a2e5d-3618-4540-88e9-ffaeae12d7e2 None admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.280903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a2e5d-3618-4540-88e9-ffaeae12d7e2 None admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.282449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a2e5d-3618-4540-88e9-ffaeae12d7e2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.284312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 17/30] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:31 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:31.285202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27486ec1-4c8b-4552-ab3c-ccf646f7d1e0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.285486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27486ec1-4c8b-4552-ab3c-ccf646f7d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.285752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27486ec1-4c8b-4552-ab3c-ccf646f7d1e0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.286972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 18/31] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:31.287716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27486ec1-4c8b-4552-ab3c-ccf646f7d1e0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:31.296546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:31.406571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27486ec1-4c8b-4552-ab3c-ccf646f7d1e0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.429445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 14/32] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Sep 3 01:13:30 2022] POST /identity/v3/roles => generated 219 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:13:31.430227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:31.447354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.448403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.448892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.449365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.464539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.467185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3184ca98-cf77-4963-8671-d96386cbf7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:13:31.487245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:31.488167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:13:31.488960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:13:31.488960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:13:31.488960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:31.491653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 19/33] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:31.493597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.494141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.494643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.517737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:31.519433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:31.585265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:31.605124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 20/34] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:31.606468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:31.682024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:31.689282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c096c2-050e-426d-9de2-2415028d1dcb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.730329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff5cb7-1d2a-4d75-819d-7bd5bb944a49 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.737763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.738492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.738775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.739046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.743620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.745284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.746973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 15/35] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:31.752692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:31.774206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.774867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.775152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.775488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.776155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:31.776461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.778639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.809263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 21/36] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:31.816011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:31.879781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:31.882004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b69da-35db-4ab3-be71-94ef0c46fb3b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.925900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bfa53-a95e-4832-817c-c00b397840c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.926607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bfa53-a95e-4832-817c-c00b397840c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.926890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bfa53-a95e-4832-817c-c00b397840c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.927165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bfa53-a95e-4832-817c-c00b397840c4 None admin] PATH_INFO: `/v3/domains/default/users/3188ed8aefba4edeb5f28b0b4a901aed/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.929556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bfa53-a95e-4832-817c-c00b397840c4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3188ed8aefba4edeb5f28b0b4a901aed, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.931259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bfa53-a95e-4832-817c-c00b397840c4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:31.942456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f1d65-9ff8-45cf-9d60-1bd318d29abd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:31.978491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3800d-edb3-4778-b5f2-7e3c6d933982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:31.979110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3800d-edb3-4778-b5f2-7e3c6d933982 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:31.979526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3800d-edb3-4778-b5f2-7e3c6d933982 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:31.979810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3800d-edb3-4778-b5f2-7e3c6d933982 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:31.980491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3800d-edb3-4778-b5f2-7e3c6d933982 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:31.981953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3800d-edb3-4778-b5f2-7e3c6d933982 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:32.058131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 16/37] 10.222.0.13 () {58 vars in 1361 bytes} [Sat Sep 3 01:13:31 2022] PUT /identity/v3/domains/default/users/3188ed8aefba4edeb5f28b0b4a901aed/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:13:32.059773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b617a-5dfb-4ee7-bc9f-a4de3c4b0fe1 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.060150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b617a-5dfb-4ee7-bc9f-a4de3c4b0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.060644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b617a-5dfb-4ee7-bc9f-a4de3c4b0fe1 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.062232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 17/38] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:32.062870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b617a-5dfb-4ee7-bc9f-a4de3c4b0fe1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:32.090856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:32.176367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b617a-5dfb-4ee7-bc9f-a4de3c4b0fe1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:32.202542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 22/39] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:13:31 2022] POST /identity/v3/projects => generated 301 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:13:32.205028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.205495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.205940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.216207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a88992-db6d-4a83-934e-4c640b8dd26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:32.217559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a88992-db6d-4a83-934e-4c640b8dd26e None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.217559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a88992-db6d-4a83-934e-4c640b8dd26e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.217837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a88992-db6d-4a83-934e-4c640b8dd26e None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.218579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a88992-db6d-4a83-934e-4c640b8dd26e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:32.220929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a88992-db6d-4a83-934e-4c640b8dd26e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:32.231720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:32.233274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:32.290858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 18/40] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Sep 3 01:13:32 2022] POST /identity/v3/roles => generated 225 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:32.293440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629741e-4491-4d82-8c78-fc0f5570bc61 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.293989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629741e-4491-4d82-8c78-fc0f5570bc61 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.294612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629741e-4491-4d82-8c78-fc0f5570bc61 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.297381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 19/41] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:32.303793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.303793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.304386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.324830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 23/42] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:32.335859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:32.337932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:32.432848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 20/43] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:32.645443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:32.674212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:32.713389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:32.730155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:32.828049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60d5b9-76ac-46bf-b7df-107604e8c895 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:32.863849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:32.864494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.864782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.865056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.865155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7270f4ff-5b38-4cda-b3ca-1513347cebcf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:32.866174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:32.867737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:32.869446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 21/44] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:32.875514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:32.900643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417e803-69a5-4be4-b6b6-17327d3a6281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:32.901189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417e803-69a5-4be4-b6b6-17327d3a6281 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:32.901467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417e803-69a5-4be4-b6b6-17327d3a6281 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:32.901746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417e803-69a5-4be4-b6b6-17327d3a6281 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:32.902455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417e803-69a5-4be4-b6b6-17327d3a6281 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:32.904635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417e803-69a5-4be4-b6b6-17327d3a6281 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:32.960114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:32.978068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d012-8944-474e-acde-b9e8eab73ac2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:33.007060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 24/45] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Sep 3 01:13:32 2022] POST /identity/v3/roles => generated 223 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:33.015453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6d037-82f4-4a11-9c2d-4181d3eaa846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:33.016341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6d037-82f4-4a11-9c2d-4181d3eaa846 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:33.017331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6d037-82f4-4a11-9c2d-4181d3eaa846 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:33.018306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6d037-82f4-4a11-9c2d-4181d3eaa846 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:33.019730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6d037-82f4-4a11-9c2d-4181d3eaa846 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:33.021282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6d037-82f4-4a11-9c2d-4181d3eaa846 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:33.094944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 22/46] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:13:32 2022] POST /identity/v3/projects => generated 312 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:13:34.051643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7614be0-9ba9-4989-93ba-e3af77e72549 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:34.052195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7614be0-9ba9-4989-93ba-e3af77e72549 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:34.052195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7614be0-9ba9-4989-93ba-e3af77e72549 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:34.053559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 25/47] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:34.058892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:34.059184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:34.059460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:34.078048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:34.079280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:34.143732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 23/48] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:34.408482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7614be0-9ba9-4989-93ba-e3af77e72549 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:34.496579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:34.509803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7614be0-9ba9-4989-93ba-e3af77e72549 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:34.543880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:34.545196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:34.545480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:34.545756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:34.556201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:34.562125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:34.563021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:34.563804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 26/49] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:34.569047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:34.629585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:34.666728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16480b13-866e-42d2-98ed-7d510f22bea5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:34.700542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:34.701662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:34.702027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:34.702307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:34.703007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:34.703315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:34.704716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:34.717017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 24/50] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:34.722512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:34.823425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d93a3-c905-4548-a6e6-8a21d7c85ff4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:34.856966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975db83-c6cf-4109-9175-37fbc17d7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:34.857608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975db83-c6cf-4109-9175-37fbc17d7687 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:34.857893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975db83-c6cf-4109-9175-37fbc17d7687 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:34.858171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975db83-c6cf-4109-9175-37fbc17d7687 None admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:34.859325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975db83-c6cf-4109-9175-37fbc17d7687 None admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:34.860805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975db83-c6cf-4109-9175-37fbc17d7687 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:34.862566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:34 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:34.867964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:34.968985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a7838e-3670-4065-bc28-3c23bbb646cc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:35.032484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d01f0-4f42-44b6-96db-72fa42cf9ff6 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:35.032772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d01f0-4f42-44b6-96db-72fa42cf9ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:35.033057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d01f0-4f42-44b6-96db-72fa42cf9ff6 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:35.034621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 28/52] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:13:35.040092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:35.040379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:35.040655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:35.059867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:35.061753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:35.073676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:35.099056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:35.107709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:35.108396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:35.108686 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:35.108962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:35.110094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:35.111598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:35.113182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 25/53] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:35.119107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:35.132721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 29/54] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:35.216568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c509bfc-22d9-41ae-967d-64f052f02f81 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:35.249036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:35.249631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:35.249913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:35.250185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:35.250936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.domain.id=default, user.id=3188ed8aefba4edeb5f28b0b4a901aed) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:35.251243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:35.252675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:35.260313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 26/55] 10.222.0.13 () {56 vars in 1322 bytes} [Sat Sep 3 01:13:35 2022] GET /identity/v3/role_assignments?user.id=3188ed8aefba4edeb5f28b0b4a901aed&scope.domain.id=default&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:35.403866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:35.503101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba927b2-9682-48e8-be4d-c4788f364755 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:35.534887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19305713-21c8-4f10-a98e-14781c4a92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:35.535518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19305713-21c8-4f10-a98e-14781c4a92fa None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:35.535798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19305713-21c8-4f10-a98e-14781c4a92fa None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:35.536070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19305713-21c8-4f10-a98e-14781c4a92fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:35.537161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19305713-21c8-4f10-a98e-14781c4a92fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:35.538654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19305713-21c8-4f10-a98e-14781c4a92fa None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:35.540211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:35.546462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:35.603923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:35.643572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077d850-12bb-44f2-b5ec-78559fe352af None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:35.677624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effd470-0801-40f3-b70a-d7c1106e5804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:35.678215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd470-0801-40f3-b70a-d7c1106e5804 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:35.678526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd470-0801-40f3-b70a-d7c1106e5804 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:35.678797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd470-0801-40f3-b70a-d7c1106e5804 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:35.679445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effd470-0801-40f3-b70a-d7c1106e5804 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:35.680908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effd470-0801-40f3-b70a-d7c1106e5804 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:35.749811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 27/57] 10.222.0.13 () {60 vars in 1140 bytes} [Sat Sep 3 01:13:35 2022] POST /identity/v3/projects => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:37.660734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bff916-3d00-4c40-a274-b4e20351b982 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:37.661241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bff916-3d00-4c40-a274-b4e20351b982 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:37.661342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bff916-3d00-4c40-a274-b4e20351b982 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:37.662665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:37.667914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:37.668317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:37.668470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:37.687930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:37.689105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:37.752855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:38.028272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bff916-3d00-4c40-a274-b4e20351b982 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:38.044354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:38.129957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bff916-3d00-4c40-a274-b4e20351b982 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:38.163644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c6019-dea1-426e-99e0-ad7cf1433d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:38.164281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c6019-dea1-426e-99e0-ad7cf1433d2c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:38.164565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c6019-dea1-426e-99e0-ad7cf1433d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:38.164838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c6019-dea1-426e-99e0-ad7cf1433d2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:38.165956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c6019-dea1-426e-99e0-ad7cf1433d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:38.167473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c6019-dea1-426e-99e0-ad7cf1433d2c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:38.169031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:38.175195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:38.215192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:38.271879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61971d-efe4-4968-8571-983a128d2fe5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:38.305237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4119d8-981f-4f30-8dd1-b63292be831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:38.305783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4119d8-981f-4f30-8dd1-b63292be831e None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:38.306064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4119d8-981f-4f30-8dd1-b63292be831e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:38.306359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4119d8-981f-4f30-8dd1-b63292be831e None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:38.307013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4119d8-981f-4f30-8dd1-b63292be831e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:38.308463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4119d8-981f-4f30-8dd1-b63292be831e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:38.429559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:13:38 2022] POST /identity/v3/users => generated 275 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:13:41.417646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c490c-ef17-4dc2-930c-194133eb15e7 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:41.418578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c490c-ef17-4dc2-930c-194133eb15e7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:41.419006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c490c-ef17-4dc2-930c-194133eb15e7 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:41.421464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:41.426997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a031ad-2ca6-4f51-a259-e2f343c008ab None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:41.427776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a031ad-2ca6-4f51-a259-e2f343c008ab None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:41.428199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a031ad-2ca6-4f51-a259-e2f343c008ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:41.459417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-80a031ad-2ca6-4f51-a259-e2f343c008ab None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:41.460885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-80a031ad-2ca6-4f51-a259-e2f343c008ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:41.477490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad28e0-18cd-4198-b1bd-4886de1d43f4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:41.477996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad28e0-18cd-4198-b1bd-4886de1d43f4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:41.478449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad28e0-18cd-4198-b1bd-4886de1d43f4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:41.480942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 34/63] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:41.486918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:41.487218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:41.487493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:41.521643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:41.522891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:41.587159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 30/64] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:41.635984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 35/65] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:41.931974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d6aea-8047-4339-a04b-189363fdd89a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:41.932355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d6aea-8047-4339-a04b-189363fdd89a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:41.932542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d6aea-8047-4339-a04b-189363fdd89a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:41.934132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 31/66] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:41.941391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:41.944416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:41.944943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:41.945519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:41.965067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:41.975510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:41.981287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:42.008952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.063040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 32/67] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:42.063996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.087506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394546e1-ba2b-441e-985e-31bcbad1452e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.124912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc968fd3-eabf-4890-81f8-c5b392b24ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.125611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc968fd3-eabf-4890-81f8-c5b392b24ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.125901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc968fd3-eabf-4890-81f8-c5b392b24ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.126178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc968fd3-eabf-4890-81f8-c5b392b24ef5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.136296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc968fd3-eabf-4890-81f8-c5b392b24ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-dc968fd3-eabf-4890-81f8-c5b392b24ef5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:42.141334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:42.142194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:42.143001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 36/68] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:42.144130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73df5-08c5-40c8-bcd9-4d572ee27150 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.144388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73df5-08c5-40c8-bcd9-4d572ee27150 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.144671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73df5-08c5-40c8-bcd9-4d572ee27150 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.146419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 37/69] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:42.147235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115aad0-eac8-496d-8106-8828d399d3c3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.147519 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115aad0-eac8-496d-8106-8828d399d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.147796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115aad0-eac8-496d-8106-8828d399d3c3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.149102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 38/70] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:42.151323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115aad0-eac8-496d-8106-8828d399d3c3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.177988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c7e29-863d-4772-8c03-c5751212e785 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.215196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5177ce-3998-4663-8049-d1297b60086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.215847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5177ce-3998-4663-8049-d1297b60086b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.216808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5177ce-3998-4663-8049-d1297b60086b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.217095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5177ce-3998-4663-8049-d1297b60086b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.228542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.228886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5177ce-3998-4663-8049-d1297b60086b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4b5177ce-3998-4663-8049-d1297b60086b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:42.238373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:42.241928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 33/71] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:42.242729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.243407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.243407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.267738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:42.269335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:42.272146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115aad0-eac8-496d-8106-8828d399d3c3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.284534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.314800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.316261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.316546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.317592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.319395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:42.320168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:42.321583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538347b2-386a-4be1-8248-e3f840fbc3b5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:42.331196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 39/72] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:42.332258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.332555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.332832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.351625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:42.352833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:42.352926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 34/73] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:42.354368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.425986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 40/74] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:42.426876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.465525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecdaa7-94d6-4982-b8bd-9c0a0ab3e6c2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.508769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.509743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.509743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.509945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.510625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:42.510936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:42.512342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:42.521960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.526489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 35/75] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:42.527377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.572487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8f4ff-14c9-4220-9a70-32fec009d3a6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.611379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.612417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.612953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.614556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.629842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:42.631393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:42.633205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 41/76] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:42.633847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.644302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef10c-1262-4e69-b62f-6c39abdd43c6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.684551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.685240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.685528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.686075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.696661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:42.702038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:42.703028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 36/77] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:42.708588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.741970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82e024-af2f-40d8-b070-ee1df6d9fb71 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.764929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.770852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.776462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.777101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.777388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.777665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.779594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:42.781060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:42.782787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 42/78] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:42.787436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.804180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae5533-73d1-4d7c-a27e-26fa97e941ab None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.840322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.840890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.841729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.842002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.843239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:42.844702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:42.846263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 37/79] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:42.847053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:42.889237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99efe0ea-1fad-4ae9-b651-9f0501f880f4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.928019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.928639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.928926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.929205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.952593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6cf0e-d36b-4d62-9928-acbdb68087dd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:42.987680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:42.988315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:42.988600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:42.988880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:42.989105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.991746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:42.999536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:43.004073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.004073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:43.004430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:43.005431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:43.006203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 38/80] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:43.006904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.099433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 43/81] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:43.100577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.112369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72efaf9-b777-4edf-a3c7-72ff498a0abf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.147498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.148113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.148399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.148677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.149350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:43.149658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.151099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.158892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 39/82] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.159661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.188043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.197174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90167532-d15c-4119-95b0-36f125e4a67d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.217280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.233539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.234258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.234568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.234847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.259486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4651b31-3837-4a03-b39a-749c1f4bc58d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.292878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.293488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.293774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.294717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.295707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.297644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.301345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.302861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.365751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 40/83] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:13:43 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:43.366544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.395352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.407888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 44/84] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:43.413546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.491320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822ff0-f376-4770-a0b6-e7680d5a093e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.510206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.533977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.534647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.534937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.535217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.535931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6cc98-c30d-4667-88cc-415f9e4ce52a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.536343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:43.536343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.537582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.547418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 41/85] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.548347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.580621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.581257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.581539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.581810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.583077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.584581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.586385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 45/86] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.588086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.667443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.670999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16333a9d-fe74-4349-bd2d-dec60563e13d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.715008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e011e11-eadf-4d0f-bb62-71f7191abb15 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.719167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.719817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.720099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.720371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.721529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.729457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.729457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 42/87] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.729457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.768864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.769509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.769789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.770325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.772052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.773570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.776459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 46/88] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.777256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.778490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.846500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593e91f-78c7-46b4-b5ac-30aeebf2347c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.884523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.885259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.885544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.885816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.886955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.888755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.890389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 43/89] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.896086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:43.897912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3543b3-b48a-45da-99a3-029bde01e828 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:43.930974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:43.937149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:43.937765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:43.938102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:43.938404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:43.939155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=d0035d46b18a48c497b26eac6fb960ad) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:43.939464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:43.941211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:43.947653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 47/90] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/role_assignments?user.id=d0035d46b18a48c497b26eac6fb960ad&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:43.948623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:44.012323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fa47b-cd16-4d21-9329-fb81163a0c2e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:44.028205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:44.052414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:44.053276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:44.053696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:44.054103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:44.171992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:44.173522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:44.198552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8d701-e658-43b9-869b-e669c07733e0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:44.260042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:44.260960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:44.261495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:44.261949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:44.263563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=3188ed8aefba4edeb5f28b0b4a901aed) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:44.264370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:44.266665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:44.275785 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 44/91] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:44.276717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:44.280925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 48/92] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:43 2022] GET /identity/v3/role_assignments?user.id=3188ed8aefba4edeb5f28b0b4a901aed&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:44.281983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:44.292094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:44.385793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e9b07-f2a9-4ad1-a2af-89213dfc168d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:44.390929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:44.439506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:44.440427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:44.441315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:44.441724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:44.499614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c245f-8aac-4332-bdf6-3046b75a6ddc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:44.571382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:44.572793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:44.573837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:44.574315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:44.577820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:44.582474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:44.585059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 49/93] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:44 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:44.591915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:44.609364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:44.611754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:44.634681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:44.794404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7a8e4-08ca-41bd-acb9-34148aa34432 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:44.806370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 45/94] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:44.815385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:44.832404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:44.845550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:44.846448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:44.851526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:44.851526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:44.851526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] RBAC: Adding query filter params (role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=d0035d46b18a48c497b26eac6fb960ad) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:44.851526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:44.851526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:44.856832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 50/95] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:44 2022] GET /identity/v3/role_assignments?user.id=d0035d46b18a48c497b26eac6fb960ad&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=3a297fae4df9419b9c53354550781a57 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:45.016899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece8a99-6cb1-4a35-97ad-2152cb90f5d6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:45.040773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:45.068283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de3ef7-5a6c-43ee-8761-2f17f41642a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:45.068862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de3ef7-5a6c-43ee-8761-2f17f41642a1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:45.069149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de3ef7-5a6c-43ee-8761-2f17f41642a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:45.069428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de3ef7-5a6c-43ee-8761-2f17f41642a1 None admin] PATH_INFO: `/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:45.075829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de3ef7-5a6c-43ee-8761-2f17f41642a1 None admin] RBAC: Authorizing `identity:get_project(project_id=13be92a099e94e3ea2e7c285f86dbeb2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:45.077720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de3ef7-5a6c-43ee-8761-2f17f41642a1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:45.079938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 46/96] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:44 2022] GET /identity/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2 => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:45.086329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:45.232089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde3193-b626-43ea-9f68-1d291ce23c15 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:45.274049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:45.274684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:45.274976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:45.275249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:45.275986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=13be92a099e94e3ea2e7c285f86dbeb2, user.id=d0035d46b18a48c497b26eac6fb960ad) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:45.276353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:45.277878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84576958-1641-4ef5-9d4d-2df1051c953b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:45.285777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 51/97] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:45 2022] GET /identity/v3/role_assignments?user.id=d0035d46b18a48c497b26eac6fb960ad&scope.project.id=13be92a099e94e3ea2e7c285f86dbeb2&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:46.000478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477c34-c136-4ff9-b08e-6449d7149fe7 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.001480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477c34-c136-4ff9-b08e-6449d7149fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.001957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3477c34-c136-4ff9-b08e-6449d7149fe7 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.003767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 47/98] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:46.009550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf08283-1c1e-4317-9057-6b16066791f0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.010091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf08283-1c1e-4317-9057-6b16066791f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.010266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf08283-1c1e-4317-9057-6b16066791f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.039744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9bf08283-1c1e-4317-9057-6b16066791f0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:46.041005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9bf08283-1c1e-4317-9057-6b16066791f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:46.093808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:46.119685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 52/99] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:46.380387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d22bf-220a-45c2-8fc0-ad356d8c8705 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.380387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d22bf-220a-45c2-8fc0-ad356d8c8705 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.380387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d22bf-220a-45c2-8fc0-ad356d8c8705 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.381068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 48/100] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:46.389933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.390224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.390595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.416422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:46.417654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:46.507518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 53/101] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:46.626624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d22bf-220a-45c2-8fc0-ad356d8c8705 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:46.754731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d22bf-220a-45c2-8fc0-ad356d8c8705 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:46.812369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6f49e-c72f-4801-9609-c2839f758484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:46.813016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f49e-c72f-4801-9609-c2839f758484 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.813303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f49e-c72f-4801-9609-c2839f758484 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.813580 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6f49e-c72f-4801-9609-c2839f758484 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.839232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6f49e-c72f-4801-9609-c2839f758484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-6ca6f49e-c72f-4801-9609-c2839f758484 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:46.844681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:46.845529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:13:46.846275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:46.848318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 49/102] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:46.853378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:46.916266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727486f-07e2-4801-b3bb-1fc86beab8ae None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.916576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727486f-07e2-4801-b3bb-1fc86beab8ae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.916822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727486f-07e2-4801-b3bb-1fc86beab8ae None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.918811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 50/103] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:13:46.932100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:46.938493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:46.938493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:46.968529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:46.972805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:47.017413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7f00d-4aaa-463b-98c1-25e2e7343e8e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.036355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:47.058302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:47.072359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:47.073002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.073288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.073567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.074243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:47.074697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:47.078830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:47.095921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 54/104] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:47.096786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:47.124608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 51/105] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:47.125515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:47.276016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6dae8-6878-4165-8388-51afec6f5f32 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.303969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ced5b8-dc51-41b6-adc2-fd6852ce4cef None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.328070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fc559-3367-48b6-a3f2-343acb627ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:47.328709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fc559-3367-48b6-a3f2-343acb627ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.329044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fc559-3367-48b6-a3f2-343acb627ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.329320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fc559-3367-48b6-a3f2-343acb627ef8 None admin] PATH_INFO: `/v3/users/47bb481d0eaa437b867b97da4bdeb77e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.356114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fc559-3367-48b6-a3f2-343acb627ef8 None admin] RBAC: Authorizing `identity:get_user(user_id=47bb481d0eaa437b867b97da4bdeb77e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:47.358995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fc559-3367-48b6-a3f2-343acb627ef8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:47.360840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 52/106] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:47 2022] GET /identity/v3/users/47bb481d0eaa437b867b97da4bdeb77e => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:47.363438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495d834-2e90-4962-bf8f-af52563af9bc None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.364979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495d834-2e90-4962-bf8f-af52563af9bc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.365262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495d834-2e90-4962-bf8f-af52563af9bc None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.366495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 53/107] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:47.368043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23625f-717a-43d0-a590-c4a171c25921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:47.368736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23625f-717a-43d0-a590-c4a171c25921 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.369018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23625f-717a-43d0-a590-c4a171c25921 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.369287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23625f-717a-43d0-a590-c4a171c25921 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.370651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23625f-717a-43d0-a590-c4a171c25921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:47.372492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23625f-717a-43d0-a590-c4a171c25921 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:47.373319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495d834-2e90-4962-bf8f-af52563af9bc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:47.375475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 55/108] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:47.379505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.379792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.380067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.414280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:47.416448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:47.435969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:47.484687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:47.521514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 56/109] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:47.526447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:47.565228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495d834-2e90-4962-bf8f-af52563af9bc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.630499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8f9a8-cb14-4e59-b732-1b1546d95a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:47.631449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8f9a8-cb14-4e59-b732-1b1546d95a8f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.631926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8f9a8-cb14-4e59-b732-1b1546d95a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.632398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8f9a8-cb14-4e59-b732-1b1546d95a8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.687411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07789ee-ad2c-40d6-97bc-353e2e8da950 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.708087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8f9a8-cb14-4e59-b732-1b1546d95a8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:47.709659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8f9a8-cb14-4e59-b732-1b1546d95a8f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:47.724536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:47.725344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.725633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.725912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.726590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:47.728171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:47.737992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:47.816107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 57/110] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:13:47 2022] POST /identity/v3/projects => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:47.817329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:47.824561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 54/111] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:47.824561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c32fa9-8925-41b2-b867-43acd48dd7ae None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:47.825054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c32fa9-8925-41b2-b867-43acd48dd7ae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:47.826116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c32fa9-8925-41b2-b867-43acd48dd7ae None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:47.833887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 55/112] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:47.833887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c32fa9-8925-41b2-b867-43acd48dd7ae None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:47.851937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:47.968425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65138620-911c-4016-8f9c-eecdc59677e8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.968608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c32fa9-8925-41b2-b867-43acd48dd7ae None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:47.996922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.007386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.008037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.008320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.008599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.021207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:48.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:48.027376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:48.028156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 56/113] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:48.029324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.033660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fc98b-f064-46e8-a0b8-d8c18289d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.034320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fc98b-f064-46e8-a0b8-d8c18289d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.034635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fc98b-f064-46e8-a0b8-d8c18289d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.034981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fc98b-f064-46e8-a0b8-d8c18289d0e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.047012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01fc98b-f064-46e8-a0b8-d8c18289d0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f01fc98b-f064-46e8-a0b8-d8c18289d0e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:48.052423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:48.053422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:48.054255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:48.054936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d01af3-d4f7-414c-9612-098ef85b6166 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.055619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d01af3-d4f7-414c-9612-098ef85b6166 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.055619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d01af3-d4f7-414c-9612-098ef85b6166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.079130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f7d01af3-d4f7-414c-9612-098ef85b6166 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:48.080331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f7d01af3-d4f7-414c-9612-098ef85b6166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:48.152484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 59/115] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:48.153890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d912-3f50-458f-bdbd-c654c2e21e18 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.154184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d912-3f50-458f-bdbd-c654c2e21e18 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.154487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79d912-3f50-458f-bdbd-c654c2e21e18 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.154872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9560990-1121-4edf-a9fb-3e22a00f1987 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.155921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 60/116] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:48.160830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79d912-3f50-458f-bdbd-c654c2e21e18 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.176552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.204033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.204756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.205041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.205317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.209303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.211076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.212739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 57/117] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.214572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.281012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.300226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79d912-3f50-458f-bdbd-c654c2e21e18 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.331649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39787b-ddec-4545-b494-cd44ecc471ae None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.347002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.347764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.348626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.349012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.350074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:48.350516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.352297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e6f3e-ad48-4a62-abe5-06e35fc3dc1a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.364811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 61/118] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.366293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.366707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.367507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.373656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.374544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.374991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.375515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.376490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:48.377023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.378767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.388274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 58/119] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.389438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:48.389802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.390681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:48.482385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 62/120] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:48.482385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.482385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.520141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44757043-366c-49f1-bfe4-e64f0747842a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.544815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.565471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.566397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.566689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.567015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.568038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] RBAC: Adding query filter params (role.id=a40cb285917842ecb5be4fd4b47cc319, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=47bb481d0eaa437b867b97da4bdeb77e) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:48.568352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.570632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.577779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 59/121] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/role_assignments?user.id=47bb481d0eaa437b867b97da4bdeb77e&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=a40cb285917842ecb5be4fd4b47cc319 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.578769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.602900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0a0fb-e332-4801-9b45-e54c66c46114 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.683806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.684613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.684969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.685445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.691268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.692883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.694897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 63/122] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.695533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.705716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f30e94-d630-4231-9825-9481ad7d8576 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.747832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.748644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.748934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.749207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.750814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.752312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.753972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 60/123] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.754778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.827015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.836828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:48.871366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622e919-6419-480e-9ac5-7b8e8debae57 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.896217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9750d-c193-4666-a75f-0588e713e823 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:48.916474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.917215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.917572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.917921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.933712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:48.937984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:48.939001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:48.939489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:48.939945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:48.940045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:48.940813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:48.941561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 64/124] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:48.942175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:48.944496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:48.946465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:48.948973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 61/125] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:48.954370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.058197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeea9f-ad9b-418c-a65f-c7f76a3a63a2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.089362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ca795-a558-4b6f-be79-95eeb64128c7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.089787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:49.101290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.102363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.102803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.104317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.104687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.107023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.108961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 65/126] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:49.114834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.122964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:49.130862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958db37f-6805-4402-979d-547715c3660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.131769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958db37f-6805-4402-979d-547715c3660a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.132051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958db37f-6805-4402-979d-547715c3660a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.132328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958db37f-6805-4402-979d-547715c3660a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.146905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958db37f-6805-4402-979d-547715c3660a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-958db37f-6805-4402-979d-547715c3660a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:49.153684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:49.154743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:49.156050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:49.158787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 62/127] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:49.159305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958db37f-6805-4402-979d-547715c3660a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.238439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827431-f3f3-473f-8fbd-839bf78a0996 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.274951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.275894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.276396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.276859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.277763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:49.278886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.280554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.291752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 66/128] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:13:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:49.292483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.296831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958db37f-6805-4402-979d-547715c3660a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.332980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.334064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.334583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.335082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce/users/3188ed8aefba4edeb5f28b0b4a901aed/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.337256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] RBAC: Authorizing `identity:create_grant(project_id=591bfc2823974e3f9466598de018ddce, user_id=3188ed8aefba4edeb5f28b0b4a901aed, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.339029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.362301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:49.402546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c8c54-2f8b-4dd7-862d-9d5981a5a3da None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.413299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 63/129] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:13:49 2022] PUT /identity/v3/projects/591bfc2823974e3f9466598de018ddce/users/3188ed8aefba4edeb5f28b0b4a901aed/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:13:49.421567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.468811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.470654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.471930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.472257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce/users/d0035d46b18a48c497b26eac6fb960ad/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.475918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] RBAC: Authorizing `identity:create_grant(project_id=591bfc2823974e3f9466598de018ddce, user_id=d0035d46b18a48c497b26eac6fb960ad, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.477450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.503340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:49.576307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177fdc8-4394-446d-87ac-b402584781da None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.593662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 67/130] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:13:49 2022] PUT /identity/v3/projects/591bfc2823974e3f9466598de018ddce/users/d0035d46b18a48c497b26eac6fb960ad/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:13:49.594579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.626190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.627208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.627651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.628118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.629053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:49.629515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.631252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.645593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 64/131] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:49.670048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.723046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa095-3cf9-4afd-86db-1acae4496c67 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.751888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:49.764414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.774445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.774445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.774445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.774445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.774445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.774445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 68/132] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:49 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:49.776619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.877177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:49.882325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049e0f7-5117-4a27-b193-73ff4d0f5da6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.905675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c25b4-fe4e-492e-826c-fcd190251b67 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:49.947685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337da39-91a1-4311-bc25-33e212da3517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.948305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337da39-91a1-4311-bc25-33e212da3517 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.948600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337da39-91a1-4311-bc25-33e212da3517 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.948900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337da39-91a1-4311-bc25-33e212da3517 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.950410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337da39-91a1-4311-bc25-33e212da3517 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.950845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:49.951643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.952004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337da39-91a1-4311-bc25-33e212da3517 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.952214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.952643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.956902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 65/133] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:49 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:49.956902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfab5ba-bec8-4fd0-8086-84b58e6f0c9d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.956902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfab5ba-bec8-4fd0-8086-84b58e6f0c9d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.956902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfab5ba-bec8-4fd0-8086-84b58e6f0c9d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:49.957256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 66/134] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:49.957668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:49.959463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:49.961659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 69/135] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:49 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:49.967748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:49.968763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:49.969618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:49.971521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:50.008957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:50.012238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:50.080469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:50.128040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 67/136] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:50.129170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:50.135965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69387e-ae3a-430a-9ba4-2d63ecfea107 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:50.207114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:50.207839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:50.208215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:50.208585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] PATH_INFO: `/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:50.209858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] RBAC: Authorizing `identity:get_project(project_id=13be92a099e94e3ea2e7c285f86dbeb2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:50.211587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:50.216838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 70/137] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:49 2022] GET /identity/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2 => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:50.224241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:50.329702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:50.338095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440a769-8d78-4b49-8887-3920da46b50e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:50.365734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7865807-dd12-4384-aa1c-f10438dc8943 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:50.400048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:50.400743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:50.401028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:50.401308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce/users/d0035d46b18a48c497b26eac6fb960ad/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:50.402265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:50.403062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:50.404631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:50.405001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] PATH_INFO: `/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2/users/d0035d46b18a48c497b26eac6fb960ad/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:50.406725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:50.407999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] RBAC: Authorizing `identity:create_grant(project_id=13be92a099e94e3ea2e7c285f86dbeb2, user_id=d0035d46b18a48c497b26eac6fb960ad, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:50.408117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] RBAC: Authorizing `identity:create_grant(project_id=591bfc2823974e3f9466598de018ddce, user_id=d0035d46b18a48c497b26eac6fb960ad, role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:50.409629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:50.409890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:50.508841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 68/138] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:13:50 2022] PUT /identity/v3/projects/591bfc2823974e3f9466598de018ddce/users/d0035d46b18a48c497b26eac6fb960ad/roles/3a297fae4df9419b9c53354550781a57 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:13:50.514596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:50.542095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 71/139] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:13:50 2022] PUT /identity/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2/users/d0035d46b18a48c497b26eac6fb960ad/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:13:50.629684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda73d8-0c0b-4be8-93a7-20a9e008f349 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:50.669570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b0ed3-73a0-4237-b928-8323b906e98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:50.670594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b0ed3-73a0-4237-b928-8323b906e98e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:50.670857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b0ed3-73a0-4237-b928-8323b906e98e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:50.671135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b0ed3-73a0-4237-b928-8323b906e98e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:50.672339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b0ed3-73a0-4237-b928-8323b906e98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:50.673892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0b0ed3-73a0-4237-b928-8323b906e98e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:50.675630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 69/140] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:50.681496 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:50.837936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69492a49-e667-4a08-9813-0ff62df5a93e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:50.879936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfcd1e-5c22-4768-b8c4-7b27f471b79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:50.880558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfcd1e-5c22-4768-b8c4-7b27f471b79c None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:50.880839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfcd1e-5c22-4768-b8c4-7b27f471b79c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:50.881123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfcd1e-5c22-4768-b8c4-7b27f471b79c None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:50.881802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfcd1e-5c22-4768-b8c4-7b27f471b79c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:50.883372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfcd1e-5c22-4768-b8c4-7b27f471b79c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:50.996274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 72/141] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:13:50 2022] POST /identity/v3/users => generated 283 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:51.059203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c70768-e2f9-412f-a666-910d471e20d3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:51.060079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c70768-e2f9-412f-a666-910d471e20d3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:51.060341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c70768-e2f9-412f-a666-910d471e20d3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:51.062079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 70/142] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:51.067357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:51.067654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:51.067929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:51.095416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:51.096654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:51.146840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:51.196498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 73/143] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:51.739253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c70768-e2f9-412f-a666-910d471e20d3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:51.774702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:51.879029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c70768-e2f9-412f-a666-910d471e20d3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:51.925384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:51.926100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:51.926409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:51.926738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:51.940715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:51.945587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:51.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:51.947219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 71/144] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:51.954316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:52.070818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b8aa1-2f27-476a-9084-75b930db09ea None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:52.110478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:52.111193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.111378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.111648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.112319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:52.112619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:52.114022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599c8a1-113f-4569-93ea-2abe54783690 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:52.126639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 74/145] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:52.132183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:52.146846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97fa6e-17f5-4bc7-bdf1-f681f269f57b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.147140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97fa6e-17f5-4bc7-bdf1-f681f269f57b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.147394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97fa6e-17f5-4bc7-bdf1-f681f269f57b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.148763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 75/146] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:52.154963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b2da0-a6ed-4d5b-ad10-2bbe22c5941d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.155241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b2da0-a6ed-4d5b-ad10-2bbe22c5941d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.155515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b2da0-a6ed-4d5b-ad10-2bbe22c5941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.174953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e13b2da0-a6ed-4d5b-ad10-2bbe22c5941d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:52.176170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e13b2da0-a6ed-4d5b-ad10-2bbe22c5941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:52.218755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:52.261044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:52.270572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 76/147] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:52.272617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46434d39-57ee-44de-9fb2-199fe37bd631 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.273007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46434d39-57ee-44de-9fb2-199fe37bd631 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.273741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46434d39-57ee-44de-9fb2-199fe37bd631 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.275403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 77/148] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:52.281472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.281834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.282183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.283289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e7542-8575-4439-87da-49f88a1c3a87 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:52.304808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:52.306126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:52.327898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:52.328831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.329290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.329746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] PATH_INFO: `/v3/users/47bb481d0eaa437b867b97da4bdeb77e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.331173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] RBAC: Authorizing `identity:get_user(user_id=47bb481d0eaa437b867b97da4bdeb77e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:52.332951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:52.335007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 72/149] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:52 2022] GET /identity/v3/users/47bb481d0eaa437b867b97da4bdeb77e => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:52.342543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:52.388067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 78/150] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:52.510056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd37a82-e5ca-4372-83f4-2c6901d92e99 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:52.569848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:52.570572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.570892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.571128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.572291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:52.573757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:52.580455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 73/151] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:52 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:52.581609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:52.694852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dbc415-8ace-41aa-a445-f9dc823d7856 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:52.706718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:52.732765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:52.733448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:52.733731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:52.734011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce/users/47bb481d0eaa437b867b97da4bdeb77e/roles/a40cb285917842ecb5be4fd4b47cc319` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:52.735988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] RBAC: Authorizing `identity:create_grant(project_id=591bfc2823974e3f9466598de018ddce, user_id=47bb481d0eaa437b867b97da4bdeb77e, role_id=a40cb285917842ecb5be4fd4b47cc319)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:52.737705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:52.801333 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 79/152] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:13:52 2022] PUT /identity/v3/projects/591bfc2823974e3f9466598de018ddce/users/47bb481d0eaa437b867b97da4bdeb77e/roles/a40cb285917842ecb5be4fd4b47cc319 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:13:52.859148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:52.878040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:52.922728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:53.077722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1600f-35d2-48f8-91ef-0e9662842e96 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:53.118410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:53.119058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.119340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.119611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.131739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:53.136907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:53.137906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:53.138708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:53.139279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 74/153] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:53.144602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:53.180400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682694c-df74-480c-bb01-d823445d9523 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:53.279520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:53.283860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be24a8-0f9f-4bb1-9a7a-9fed3f41130a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:53.309350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:53.310010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.310294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.310594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.337072 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:53.346153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:53.348042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:53.349381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:53.354361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 80/154] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 474 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:13:53.358224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:53.383170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:53.383734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.384020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.384761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.385467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:53.386077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:53.387869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:53.397367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:53.407522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 75/155] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:53.420872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:53.589308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43e866-2d50-42f3-888c-1289b03e49eb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:53.622569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965d7a1-649a-4a83-86d5-6c0d9ebc9ae2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:53.653294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:53.653839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.654261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.654579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.655260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:53.655719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:53.657148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a15cde-ddd7-40e5-b72d-8d7e0dee3d33 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:53.671715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbda5fc-fae6-45f5-be76-7841cd0db09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:53.672377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbda5fc-fae6-45f5-be76-7841cd0db09b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.672583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 81/156] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:53.672925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbda5fc-fae6-45f5-be76-7841cd0db09b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.673204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbda5fc-fae6-45f5-be76-7841cd0db09b None admin] PATH_INFO: `/v3/users/3188ed8aefba4edeb5f28b0b4a901aed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.673861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0079f10-1921-471b-87e6-2723f0f9b6a8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.674151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0079f10-1921-471b-87e6-2723f0f9b6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.674499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0079f10-1921-471b-87e6-2723f0f9b6a8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.675006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbda5fc-fae6-45f5-be76-7841cd0db09b None admin] RBAC: Authorizing `identity:get_user(user_id=3188ed8aefba4edeb5f28b0b4a901aed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:53.675998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 82/157] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:53.676479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbda5fc-fae6-45f5-be76-7841cd0db09b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:53.678077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 76/158] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:53 2022] GET /identity/v3/users/3188ed8aefba4edeb5f28b0b4a901aed => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:53.680061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0079f10-1921-471b-87e6-2723f0f9b6a8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:53.683971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:53.684798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:53.685066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:53.740507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:53.741734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:53.758781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:53.836767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:53.868928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 77/159] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:53.869814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:53.919753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0079f10-1921-471b-87e6-2723f0f9b6a8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:54.014471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937f249-4800-4cfc-80db-30f47f9ef765 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:54.015098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:54.015766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.016055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.016337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.017564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:54.019093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:54.021335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 83/160] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:53 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:54.027335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:54.075633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:54.076266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.076550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.076847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.089729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:54.176055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:54.177434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:54.262278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352ace-5c16-42b7-958c-1e88b3153536 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:54.341144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d9f68-ebb5-45b4-9f3e-1984558a5ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:54.341904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d9f68-ebb5-45b4-9f3e-1984558a5ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.342191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d9f68-ebb5-45b4-9f3e-1984558a5ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.342491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d9f68-ebb5-45b4-9f3e-1984558a5ad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.374386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:54.388554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 78/161] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:54.396289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:54.472927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d9f68-ebb5-45b4-9f3e-1984558a5ad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:54.474781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d9f68-ebb5-45b4-9f3e-1984558a5ad2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:54.483845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:54.624325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 84/162] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:13:54.630367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca2fc-9f00-4025-b68e-054f84ee902f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.630869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca2fc-9f00-4025-b68e-054f84ee902f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.631335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ca2fc-9f00-4025-b68e-054f84ee902f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.632807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 85/163] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:54.633771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca2fc-9f00-4025-b68e-054f84ee902f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:54.648314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf43baa-9c43-4e4c-be6c-335f545999e4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:54.724653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:54.751075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:54.751732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.752012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.752281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.759051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:54.760527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:54.763141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 79/164] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:54 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:54.764123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:54.891028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ca2fc-9f00-4025-b68e-054f84ee902f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:54.921354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebd729c-6fda-49bf-a01a-db297c59da1c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:54.932804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:54.957896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d25b14-891b-429e-bf74-6e9df980cb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:54.958899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d25b14-891b-429e-bf74-6e9df980cb68 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.960154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d25b14-891b-429e-bf74-6e9df980cb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.960579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d25b14-891b-429e-bf74-6e9df980cb68 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.961868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d25b14-891b-429e-bf74-6e9df980cb68 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:54.963492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d25b14-891b-429e-bf74-6e9df980cb68 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:54.965750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 80/165] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:54 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:54.967156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.967587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.968000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.983741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:54.984314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:54.984595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:54.984864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:54.987782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:54.989129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:55.023006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:55.028761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:55.029814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 86/166] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:55.031036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:55.088608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 81/167] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:55.089705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:55.243370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb762f24-f375-4598-9d02-f7fe682d29ce None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:55.254169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:55.293240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b42af-817d-41ff-b117-a6d288f490de None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:55.303612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:55.304443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.304876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:55.305010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.305393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.306275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=d0035d46b18a48c497b26eac6fb960ad) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:55.306742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:55.308337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:55.323505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 82/168] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:55 2022] GET /identity/v3/role_assignments?user.id=d0035d46b18a48c497b26eac6fb960ad&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:55.324538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:55.404350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:55.404980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.405256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.405522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.406260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=3188ed8aefba4edeb5f28b0b4a901aed) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:55.406587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:55.408279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd5692-a4bb-4d6e-bcd1-127fc17091b7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:55.432406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 87/169] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:55 2022] GET /identity/v3/role_assignments?user.id=3188ed8aefba4edeb5f28b0b4a901aed&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:55.493791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7548d-ee42-449c-8ae8-b1431ac7a122 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.493970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7548d-ee42-449c-8ae8-b1431ac7a122 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.494282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7548d-ee42-449c-8ae8-b1431ac7a122 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.496012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 88/170] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:55.496553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d9ff3-191a-4613-b2e9-b2ffaada0897 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:55.503875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.504154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.504430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.542595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:55.544477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:55.555107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:55.555994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.556422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.556842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.557679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:55.558155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:55.559845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:55.569492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 83/171] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:55.570631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:55.644656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:55.728747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 89/172] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:55.729635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:55.808775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf87731-8f34-4883-9eac-66b6d40c1e80 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:55.889393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521837a-8556-4616-99ce-464bcd9d7472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:55.890042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521837a-8556-4616-99ce-464bcd9d7472 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.890321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521837a-8556-4616-99ce-464bcd9d7472 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.890620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521837a-8556-4616-99ce-464bcd9d7472 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.910874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521837a-8556-4616-99ce-464bcd9d7472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4521837a-8556-4616-99ce-464bcd9d7472 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:55.916335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:55.917210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:55.918069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 84/173] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:13:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:55.919424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2877-1a56-4bc5-bed0-3cf2239b402f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.919424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2877-1a56-4bc5-bed0-3cf2239b402f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.920083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb2877-1a56-4bc5-bed0-3cf2239b402f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.927279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 85/174] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:55.933385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:55.933679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:55.933948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:55.954908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:55.956626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:55.968485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:55.980447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad1f1b-c1f1-4f94-a388-991b2090a57e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.048744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 86/175] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:13:56.054629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.094719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.099641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.100310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.100595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.100873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.107445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:56.109147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:56.111603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 90/176] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:55 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:56.116336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.225863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee3879-cae5-4243-8585-6f05df4ef1e8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.275640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.276649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.277149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.277613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.280466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:56.280985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:56.282600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:56.294272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 87/177] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:56.295823 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.329208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906a1a5-eefc-4f27-bb91-2cfc780325d8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.371756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8725b2-a428-4d71-a0cc-a9ce54626616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.372380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8725b2-a428-4d71-a0cc-a9ce54626616 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.372659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8725b2-a428-4d71-a0cc-a9ce54626616 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.372927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8725b2-a428-4d71-a0cc-a9ce54626616 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.399721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.411791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.415994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fbcd4-4ac5-44aa-bdfc-9c43260fefb4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.445994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8725b2-a428-4d71-a0cc-a9ce54626616 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:56.447630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8725b2-a428-4d71-a0cc-a9ce54626616 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:56.451701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.452511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.452793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.453064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.463651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:56.469294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:56.470176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:56.471000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 88/178] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:56.475906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.560221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 91/179] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:13:56.561541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0191e9a-abc6-44ae-b681-8bebd380724c None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.561836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0191e9a-abc6-44ae-b681-8bebd380724c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.562109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0191e9a-abc6-44ae-b681-8bebd380724c None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.563522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 92/180] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:56.564144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0191e9a-abc6-44ae-b681-8bebd380724c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.589865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7438952-0812-4d02-8d0e-0cb7abc57738 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.630235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.630909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.631187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.631461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] PATH_INFO: `/v3/users/d0035d46b18a48c497b26eac6fb960ad` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.632757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] RBAC: Authorizing `identity:get_user(user_id=d0035d46b18a48c497b26eac6fb960ad)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:56.634442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:56.640859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.641096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 89/181] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/users/d0035d46b18a48c497b26eac6fb960ad => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:56.641096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.657419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.679107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0191e9a-abc6-44ae-b681-8bebd380724c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.712905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028d785-33a6-42e0-9a90-d3f9b79b05ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.713541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028d785-33a6-42e0-9a90-d3f9b79b05ba None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.713820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028d785-33a6-42e0-9a90-d3f9b79b05ba None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.714087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028d785-33a6-42e0-9a90-d3f9b79b05ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.725898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028d785-33a6-42e0-9a90-d3f9b79b05ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b028d785-33a6-42e0-9a90-d3f9b79b05ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:56.730920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:56.732433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:13:56.733262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 93/182] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:56.735078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.735366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.735643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.742803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8004f60-8405-4433-a7f1-138aa9148c28 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.753279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:56.754500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:56.776584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.777190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.777471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.777742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.779085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:56.779390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:56.780786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:56.788792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 90/183] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:56.789434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.826045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 94/184] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:56.826960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.890050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2019c-4001-481d-a6e2-f69bc95f5dd0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.916802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.923711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da666bca-97a1-4a3f-8a83-aea1cd8363f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.925475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da666bca-97a1-4a3f-8a83-aea1cd8363f8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.925475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da666bca-97a1-4a3f-8a83-aea1cd8363f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.925475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da666bca-97a1-4a3f-8a83-aea1cd8363f8 None admin] PATH_INFO: `/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.926396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da666bca-97a1-4a3f-8a83-aea1cd8363f8 None admin] RBAC: Authorizing `identity:get_project(project_id=13be92a099e94e3ea2e7c285f86dbeb2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:56.927868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da666bca-97a1-4a3f-8a83-aea1cd8363f8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:56.938382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 91/185] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/projects/13be92a099e94e3ea2e7c285f86dbeb2 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:56.938556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173fe14-0e4c-40b1-b5ea-7ab0c2650038 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:56.938655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c975-5d22-45c4-846a-05fdc14e2254 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.938655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c975-5d22-45c4-846a-05fdc14e2254 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.938655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1c975-5d22-45c4-846a-05fdc14e2254 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:56.938655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 92/186] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:13:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:13:56.938655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c975-5d22-45c4-846a-05fdc14e2254 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:56.938655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:56.975087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:56.975704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:56.976027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:56.976367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.045666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.046922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.056167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1c975-5d22-45c4-846a-05fdc14e2254 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.094294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.094940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.095219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.095487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.096270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:57.096682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.098175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.104024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.106820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 93/187] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:57.107620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.116376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.152150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 95/188] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:57.152927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.213970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5adda-c4c0-4416-8ae9-402355b5c9e4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.249127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55505b6e-41dd-41c3-b12a-399cc6684581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.249507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17b137-cd9f-425b-9c9e-4c26bb7e6ce6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.249754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55505b6e-41dd-41c3-b12a-399cc6684581 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.250036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55505b6e-41dd-41c3-b12a-399cc6684581 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.250309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55505b6e-41dd-41c3-b12a-399cc6684581 None admin] PATH_INFO: `/v3/users/e060a7db58524858b7c229656d25097a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.263794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55505b6e-41dd-41c3-b12a-399cc6684581 None admin] RBAC: Authorizing `identity:get_user(user_id=e060a7db58524858b7c229656d25097a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.265424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55505b6e-41dd-41c3-b12a-399cc6684581 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.267261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 94/189] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/users/e060a7db58524858b7c229656d25097a => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:57.268335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.268615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.268889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.282947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.283561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.283839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.284109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.284842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=13be92a099e94e3ea2e7c285f86dbeb2, user.id=d0035d46b18a48c497b26eac6fb960ad) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:57.285150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.286631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.287236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:13:57.288863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:13:57.293000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 96/190] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/role_assignments?user.id=d0035d46b18a48c497b26eac6fb960ad&scope.project.id=13be92a099e94e3ea2e7c285f86dbeb2&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:57.294665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.320660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.368058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.377910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 95/191] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:13:57.387638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.414076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00219a8-76dd-41ed-a5fa-08ba25de1208 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.460674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.462300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.462300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.462300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.463567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.465054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.467200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 97/192] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:57.467879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.536874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde6c0-6711-4a6c-bb4d-a6b52f3ef199 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.579030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.579708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.579992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.580266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] PATH_INFO: `/v3/users/e060a7db58524858b7c229656d25097a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.581424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] RBAC: Authorizing `identity:get_user(user_id=e060a7db58524858b7c229656d25097a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.582997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.584525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 96/193] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/users/e060a7db58524858b7c229656d25097a => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:57.585800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.591159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.604522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa863b-9b73-4fef-b8af-c8558ca7f821 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.640549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.641208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.641489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.641762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.642912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.644395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.646412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 98/194] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:57.647043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.656318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.695926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d0356-4a8d-4268-bded-c72ead259fce None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.734348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.734984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.735258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.735524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.752866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d6768-2345-443e-9835-e04ad0fbba70 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:57.786608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:57.787212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:57.787489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:57.787756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:57.788411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.789941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.805821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:57.807323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:57.866380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:57.891616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 99/195] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:13:57 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:13:57.892456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.908727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 97/196] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:57.909496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:57.909650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.011683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf786e7-57fe-4995-beaf-1e0b3412959a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.022919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1fc04-f05f-4176-ab1e-610682649b26 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.064148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.064773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.065057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.065326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.071077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.071700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.071980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.072329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.073079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] RBAC: Adding query filter params (role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=d0035d46b18a48c497b26eac6fb960ad) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:58.073391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.075974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.083346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 98/197] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/role_assignments?user.id=d0035d46b18a48c497b26eac6fb960ad&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=3a297fae4df9419b9c53354550781a57 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:58.085627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.123933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.161524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.163851 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.187033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.241268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028feb58-b2d1-4266-9a04-f8f8746ae7bc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.289198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.289771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.290054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.290332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.302784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:13:58.307958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:13:58.308848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:13:58.310531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 99/198] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:13:58.311192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 100/199] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:58.312012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.315772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.427762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.436041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a609322-ba30-4735-bddf-ade096122210 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.449556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad4878-c966-46dc-80f6-ee304ca588ac None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.452014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.479006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.480008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.480514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.481131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.482545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.484246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.490653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 101/200] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:58.491788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.494611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.495258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.495542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.495815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.498069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.498844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.500520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 100/201] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:58.505998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.603770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea189798-affb-4551-b7c9-7bea7c8a6768 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.625433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888b06c-2b37-41eb-a73b-ecf70246f252 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.645391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.646018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.646297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.646664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.647348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:58.647650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.649430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.663051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 102/202] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:58.663051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.673108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.674029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.674560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.675031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.675977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] RBAC: Adding query filter params (role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=e060a7db58524858b7c229656d25097a) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:58.676496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.678152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.690765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 101/203] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/role_assignments?user.id=e060a7db58524858b7c229656d25097a&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=3a297fae4df9419b9c53354550781a57 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:58.692059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:58.707416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.729248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:58.770820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668ff5d-861d-4b7f-917a-880dea5082f2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.815226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.815869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.816151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.816424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.817249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=e060a7db58524858b7c229656d25097a) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:58.817558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.819275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.831074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 103/204] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/role_assignments?user.id=e060a7db58524858b7c229656d25097a&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:58.920875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e501f0-c93e-429c-a69a-39ce5825a5e9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:58.960708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:58.961362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:58.961648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:58.961923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] PATH_INFO: `/v3/users/47bb481d0eaa437b867b97da4bdeb77e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:58.963152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] RBAC: Authorizing `identity:get_user(user_id=47bb481d0eaa437b867b97da4bdeb77e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:58.964648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:58.967355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 102/205] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/users/47bb481d0eaa437b867b97da4bdeb77e => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:13:58.975547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:59.115921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cdefe-0c2a-43f5-8398-75f1f0c30766 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:59.127178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:59.167636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:59.168264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:59.169188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:59.169555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:59.301207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:59.303156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:59.445532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 104/206] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:13:59.452391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:59.607461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:59.620757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebde19e-3051-47b7-b9da-c33434e3c40a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:59.673011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8973d-bd69-45b3-91d0-128164b3ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:59.673672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8973d-bd69-45b3-91d0-128164b3ee11 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:59.673955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8973d-bd69-45b3-91d0-128164b3ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:59.674227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8973d-bd69-45b3-91d0-128164b3ee11 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:59.675420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8973d-bd69-45b3-91d0-128164b3ee11 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:59.677210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8973d-bd69-45b3-91d0-128164b3ee11 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:59.679476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 103/207] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:13:59 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:13:59.686113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:13:59.694644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:13:59.845229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f81b53-49c8-4287-95e1-ac23701954ee None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:13:59.908835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:13:59.909795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:13:59.910287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:13:59.910794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:13:59.911733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] RBAC: Adding query filter params (role.id=a40cb285917842ecb5be4fd4b47cc319, scope.project.id=591bfc2823974e3f9466598de018ddce, user.id=47bb481d0eaa437b867b97da4bdeb77e) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:13:59.912241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:13:59.914021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff945b-fe7c-44c0-9817-426452580c44 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:13:59.925743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 105/208] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:13:59 2022] GET /identity/v3/role_assignments?user.id=47bb481d0eaa437b867b97da4bdeb77e&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=a40cb285917842ecb5be4fd4b47cc319 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:00.463958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436aca3-6e94-42fc-8544-8947f326022e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:00.464491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436aca3-6e94-42fc-8544-8947f326022e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:00.464491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436aca3-6e94-42fc-8544-8947f326022e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:00.466718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 104/209] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:00.471703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad428268-38d8-478a-84d1-6c4ce92020f0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:00.472117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad428268-38d8-478a-84d1-6c4ce92020f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:00.472308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad428268-38d8-478a-84d1-6c4ce92020f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:00.491521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ad428268-38d8-478a-84d1-6c4ce92020f0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:00.492729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ad428268-38d8-478a-84d1-6c4ce92020f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:00.565424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:00.571845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 106/210] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:00.671711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd7cd-e9c5-4e81-8603-b8daf0b609ab None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:00.672026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd7cd-e9c5-4e81-8603-b8daf0b609ab None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:00.672266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd7cd-e9c5-4e81-8603-b8daf0b609ab None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:00.673877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 105/211] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:00.679416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:00.679766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:00.680051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:00.700536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:00.701756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:00.774422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 107/212] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:00.856482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd7cd-e9c5-4e81-8603-b8daf0b609ab None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:00.962961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd7cd-e9c5-4e81-8603-b8daf0b609ab None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.007212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.007862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.008142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.008413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.009519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:01.011022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:01.013505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 106/213] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:01.018722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.102714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.131907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959465c2-5c04-4c78-8bc0-225202d445b4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.136612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:01.164145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:01.170218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c373ca-0f59-45f3-a845-b72f3f02e74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.170894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c373ca-0f59-45f3-a845-b72f3f02e74c None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.171227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c373ca-0f59-45f3-a845-b72f3f02e74c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.171436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c373ca-0f59-45f3-a845-b72f3f02e74c None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.172095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c373ca-0f59-45f3-a845-b72f3f02e74c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:01.173598 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c373ca-0f59-45f3-a845-b72f3f02e74c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:01.220253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d78378-c921-4023-90c3-90ec6bf61c19 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.254500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510462-05bd-4cb3-b7f7-6d345c1529cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.255128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510462-05bd-4cb3-b7f7-6d345c1529cd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.255403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510462-05bd-4cb3-b7f7-6d345c1529cd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.255672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510462-05bd-4cb3-b7f7-6d345c1529cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.266185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510462-05bd-4cb3-b7f7-6d345c1529cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5d510462-05bd-4cb3-b7f7-6d345c1529cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:01.271274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:01.272216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 107/214] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:01.275268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88ae2-0bbc-4f94-870d-464daac76eb3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.275999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88ae2-0bbc-4f94-870d-464daac76eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.275999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf88ae2-0bbc-4f94-870d-464daac76eb3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.275999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 108/215] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:01.281415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88ae2-0bbc-4f94-870d-464daac76eb3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.286453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 108/216] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:01 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:01.286453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d4ea8-5931-4a68-85ad-7da9e1b10b71 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.286453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d4ea8-5931-4a68-85ad-7da9e1b10b71 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.286934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d4ea8-5931-4a68-85ad-7da9e1b10b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.314914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b47d4ea8-5931-4a68-85ad-7da9e1b10b71 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:01.316459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b47d4ea8-5931-4a68-85ad-7da9e1b10b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:01.393513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:01.414387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 109/217] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:01.414387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc24716-db24-48ad-baa8-f9ec8d1256f1 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.414387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc24716-db24-48ad-baa8-f9ec8d1256f1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.414387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc24716-db24-48ad-baa8-f9ec8d1256f1 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.414387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 110/218] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:01.416175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.416505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.416738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.425288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf88ae2-0bbc-4f94-870d-464daac76eb3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.436827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:01.438266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:01.477386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.478632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.479051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.479479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.480294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:01.481447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:01.483010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:01.494077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 109/219] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:01.500914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.536074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 111/220] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:01.619584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f378-212c-49e8-93ce-4a0b00eb4db2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.657215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.657788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.658070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.658365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] PATH_INFO: `/v3/users/8f50fcdc926e40dc89c2f2fec4c37ea6` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.676134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] RBAC: Authorizing `identity:get_user(user_id=8f50fcdc926e40dc89c2f2fec4c37ea6)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:01.677647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:01.679537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 110/221] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:01 2022] GET /identity/v3/users/8f50fcdc926e40dc89c2f2fec4c37ea6 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:01.685785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.718207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.721075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:01.745620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:01.819673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752462c-3d8e-49d0-a66f-29196cd73e9c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.837833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc964d4-260d-4977-91f1-31c27f2e8f6a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:01.873842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.874500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.874778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.875045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.882859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:01.883835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:01.884311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:01.884787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:01.896446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:01.901584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:01.902513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:01.903891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:01.906674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 111/222] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:01.907816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:01.947208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:01.948691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:01.975003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:01.996999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.015782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc337d5-43d3-4efb-836b-c39abe483ef2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.052618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.053418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.053802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.054194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.055919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 112/223] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:02.056947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.070672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:02.075957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:02.076836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 112/224] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:02.082395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.171216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e4965-08f3-4e82-a8e2-15923ed92c2c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.182622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999be6d0-d645-49c8-9215-30212f37facb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.206604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.207482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.207902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.208313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.209117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:02.209569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.211129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.218099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.219952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 113/225] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.221112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.226718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.227398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.227685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.227961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.231914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.233570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.234375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.242464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 113/226] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.242464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.323544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcf89c-f35a-41a2-913c-f1ba98099090 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.351268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497309f5-3951-4c83-9e54-4a1850eb6c61 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.358719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.359408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.359757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.360098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.360826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:02.361192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.362686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.375756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 114/227] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.376747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.389829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.390521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.390807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.391082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] PATH_INFO: `/v3/users/e060a7db58524858b7c229656d25097a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.392425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] RBAC: Authorizing `identity:get_user(user_id=e060a7db58524858b7c229656d25097a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.393976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.400924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 114/228] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/users/e060a7db58524858b7c229656d25097a => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.401751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.447878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.471322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.479468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b37d9f-0e77-4a18-a576-bb8c2640b866 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.514862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.515583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.516016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.516356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.517248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=8f50fcdc926e40dc89c2f2fec4c37ea6) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:02.517628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.519180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.521722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e48d9-df5a-4641-be93-cec03d07e08a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.525622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 115/229] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/role_assignments?user.id=8f50fcdc926e40dc89c2f2fec4c37ea6&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.526695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.575302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.576404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.576839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.577248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] PATH_INFO: `/v3/users/e060a7db58524858b7c229656d25097a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.578831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] RBAC: Authorizing `identity:get_user(user_id=e060a7db58524858b7c229656d25097a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.580631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.587185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 115/230] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/users/e060a7db58524858b7c229656d25097a => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.607831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.645913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb2b79-4ffd-49b0-8d5c-a1ce56d2c49c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.682919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.683676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.684601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.684947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.686197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.687869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.690153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 116/231] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.696222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.701963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.803488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b8c25-1155-43c3-83b6-71a85600a899 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.837807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37348-2822-4e9e-835e-1b66cd7dffa3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:02.853974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:02.864103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.864740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.865016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.865286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.866432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.868248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:02.870078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 116/232] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:02 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:02.876964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:02.935037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f147-be14-4ffa-aa65-30de3482a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:02.936568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f147-be14-4ffa-aa65-30de3482a03d None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:02.936847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f147-be14-4ffa-aa65-30de3482a03d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:02.937116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f147-be14-4ffa-aa65-30de3482a03d None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889/users/e060a7db58524858b7c229656d25097a/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:02.943058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341f147-be14-4ffa-aa65-30de3482a03d None admin] RBAC: Authorizing `identity:create_grant(project_id=314c6a4b7a99410c99ac353aff468889, user_id=e060a7db58524858b7c229656d25097a, role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:02.945862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341f147-be14-4ffa-aa65-30de3482a03d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:03.005626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ec08-cdd7-432e-a0eb-242a30f89c87 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:03.054406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 117/233] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:02 2022] PUT /identity/v3/projects/314c6a4b7a99410c99ac353aff468889/users/e060a7db58524858b7c229656d25097a/roles/3a297fae4df9419b9c53354550781a57 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:14:03.092953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9530e76-dbbd-49cd-9749-17a1c1c3fa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:03.093924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9530e76-dbbd-49cd-9749-17a1c1c3fa80 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:03.094363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9530e76-dbbd-49cd-9749-17a1c1c3fa80 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:03.094804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9530e76-dbbd-49cd-9749-17a1c1c3fa80 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889/users/e060a7db58524858b7c229656d25097a/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:03.097022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9530e76-dbbd-49cd-9749-17a1c1c3fa80 None admin] RBAC: Authorizing `identity:create_grant(project_id=314c6a4b7a99410c99ac353aff468889, user_id=e060a7db58524858b7c229656d25097a, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:03.098732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9530e76-dbbd-49cd-9749-17a1c1c3fa80 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:03.197095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 117/234] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:02 2022] PUT /identity/v3/projects/314c6a4b7a99410c99ac353aff468889/users/e060a7db58524858b7c229656d25097a/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:14:03.672972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7adf-37c8-4c43-99e8-0e815f8df7ff None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:03.673531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7adf-37c8-4c43-99e8-0e815f8df7ff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:03.673531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d7adf-37c8-4c43-99e8-0e815f8df7ff None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:03.675182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 118/235] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:03.680086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef005c27-d1bb-422d-8be4-2061871562a8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:03.680491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef005c27-d1bb-422d-8be4-2061871562a8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:03.680696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef005c27-d1bb-422d-8be4-2061871562a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:03.707399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ef005c27-d1bb-422d-8be4-2061871562a8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:03.708622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ef005c27-d1bb-422d-8be4-2061871562a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:03.762987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:03.785822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 118/236] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:03.825615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf26ea-4122-476e-b938-746af721c854 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:03.826314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf26ea-4122-476e-b938-746af721c854 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:03.826813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf26ea-4122-476e-b938-746af721c854 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:03.828659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 119/237] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:03.833754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:03.834607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:03.834607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:03.860060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:03.861247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:03.951454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 119/238] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:04.192854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf26ea-4122-476e-b938-746af721c854 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:04.262684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:04.309640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf26ea-4122-476e-b938-746af721c854 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:04.328950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:04.358885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:04.359527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.359803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.360066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.381687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:04.387398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:04.388491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:04.389285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 120/239] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:04.396219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:04.518777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c94948-856b-4443-b6c7-a653f457d0e1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:04.547277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:04.570393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:04.570393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.570393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.570393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.571530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:04.573131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:04.575449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 120/240] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:04.581007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:04.598550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e49ebb-8d08-43fc-b74d-fdc94830f668 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:04.649489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:04.688410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:04.689175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.689558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.689902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.690696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:04.691068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:04.692600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:04.694619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123159f-2766-4559-88d9-0fb273b27bf6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:04.710187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 121/241] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:04.716428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:04.738229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2112384-e8fc-4717-86c5-59aedae5081f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:04.739174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2112384-e8fc-4717-86c5-59aedae5081f None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.740780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2112384-e8fc-4717-86c5-59aedae5081f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.741242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2112384-e8fc-4717-86c5-59aedae5081f None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.742757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2112384-e8fc-4717-86c5-59aedae5081f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:04.744559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2112384-e8fc-4717-86c5-59aedae5081f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:04.840830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 121/242] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:04 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:14:04.860666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2a5a7-901d-4517-ae2e-a12bf055b4a3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:04.910151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:04.910843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.911128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.911400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] PATH_INFO: `/v3/users/480d632482144cdfb663ba1a25be8c75` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.927290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f0192-4dc4-4f6a-a477-51b4f9d1fde4 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.927574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f0192-4dc4-4f6a-a477-51b4f9d1fde4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.927849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f0192-4dc4-4f6a-a477-51b4f9d1fde4 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.931423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 122/243] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:04.939249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:04.939532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:04.939804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:04.943653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] RBAC: Authorizing `identity:get_user(user_id=480d632482144cdfb663ba1a25be8c75)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:04.946471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:04.950210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 122/244] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:04 2022] GET /identity/v3/users/480d632482144cdfb663ba1a25be8c75 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:04.955996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:04.974613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:04.975837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:05.035728 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:05.041672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:05.097056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 123/245] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:05.109951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47280a-5dc7-4607-8471-7eaf1eb06b1a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:05.148030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:05.148648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.148923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.149189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.222592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:05.224096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:05.269324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:05.337584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 123/246] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:05.343136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:05.403776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:05.469989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c535c5a-8276-4e1c-bba5-0fe14727100b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:05.512396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771257d7-1dae-4114-8996-61f984414829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:05.513179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771257d7-1dae-4114-8996-61f984414829 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.513501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771257d7-1dae-4114-8996-61f984414829 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.513772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771257d7-1dae-4114-8996-61f984414829 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.515092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771257d7-1dae-4114-8996-61f984414829 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:05.516552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771257d7-1dae-4114-8996-61f984414829 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:05.518670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 124/247] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:05 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:05.529414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771257d7-1dae-4114-8996-61f984414829 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:05.535524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b5d63-908a-4a2a-ad05-e23056ce91f9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:05.578409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:05.581259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:05.581908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.582187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.582491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.599441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:05.606841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:05.607776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:05.608553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 124/248] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:05.615601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:05.621614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:05.659781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771257d7-1dae-4114-8996-61f984414829 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:05.707525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:05.708142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.708395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.708666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.709403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] RBAC: Adding query filter params (role.id=a40cb285917842ecb5be4fd4b47cc319, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=480d632482144cdfb663ba1a25be8c75) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:05.709707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:05.711727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96415c46-3f48-42d2-99ce-d813e2111520 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:05.720561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 125/249] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:05 2022] GET /identity/v3/role_assignments?user.id=480d632482144cdfb663ba1a25be8c75&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=a40cb285917842ecb5be4fd4b47cc319 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:05.721866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada165f-88b1-4812-a18e-e718eddd7d8a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.722157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada165f-88b1-4812-a18e-e718eddd7d8a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.722456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada165f-88b1-4812-a18e-e718eddd7d8a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.724007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 126/250] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:05.725447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d022a-adcc-442f-88a4-4b0236124802 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.725881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d022a-adcc-442f-88a4-4b0236124802 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.726292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d022a-adcc-442f-88a4-4b0236124802 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.733350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 127/251] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:05.735064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6b5a-612d-4e06-9484-b6a884d8d809 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.735521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6b5a-612d-4e06-9484-b6a884d8d809 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.736209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd6b5a-612d-4e06-9484-b6a884d8d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.759387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-09fd6b5a-612d-4e06-9484-b6a884d8d809 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:05.760866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-09fd6b5a-612d-4e06-9484-b6a884d8d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:05.776435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b48437-d6c5-4fcc-b8df-8d6b496f6840 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:05.856916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:05.857901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.858423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.858945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.859835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:05.860339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:05.860443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 128/252] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:05.866363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:05.866363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:05.866363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:05.867357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:05.883258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:05.884801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:05.886740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 125/253] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:05.893345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:05.914601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:05.994433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 129/254] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:06.019772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:06.063737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9328a-64e3-48dc-b51b-0a2b937167ae None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:06.111901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:06.112915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:06.113388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:06.113881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] PATH_INFO: `/v3/users/8f50fcdc926e40dc89c2f2fec4c37ea6` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:06.115224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] RBAC: Authorizing `identity:get_user(user_id=8f50fcdc926e40dc89c2f2fec4c37ea6)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:06.116870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:06.118687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 126/255] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:05 2022] GET /identity/v3/users/8f50fcdc926e40dc89c2f2fec4c37ea6 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:06.124569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:06.235342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6e805-b8ca-44fa-850e-5201e00a9eee None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:06.281208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:06.282183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:06.282716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:06.283193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:06.287064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:06.288726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:06.290921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 130/256] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:06 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:06.297062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:06.319478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:06.377711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:06.509503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da496795-f5eb-40ad-ab48-9159cd7ddc98 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:06.519297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba4756-c466-4e16-8b2e-df8955130306 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:06.566998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:06.574784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:06.576791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:06.577904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:06.578887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889/users/8f50fcdc926e40dc89c2f2fec4c37ea6/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:06.583033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] RBAC: Authorizing `identity:create_grant(project_id=314c6a4b7a99410c99ac353aff468889, user_id=8f50fcdc926e40dc89c2f2fec4c37ea6, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:06.585142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:06.586856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:06.589843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:06.590818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:06.591521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:06.603633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:06.609673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:06.614680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 131/257] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:06.615665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:06.675270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 127/258] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:06 2022] PUT /identity/v3/projects/314c6a4b7a99410c99ac353aff468889/users/8f50fcdc926e40dc89c2f2fec4c37ea6/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:06.676411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:06.743128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:06.764159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907b135-07a5-4a32-a8a9-9f46898bcbcf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:06.846129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:06.847052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:06.847525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:06.847938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:06.875730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:06.881119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:06.882194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:06.883155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:06.890557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c1faf-7dbc-4685-b904-b57d7fb1e159 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:06.891451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 132/259] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:06.897129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:06.943395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:06.944225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:06.944604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:06.947160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:06.947160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:06.947160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:06.947728 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:06.957493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 128/260] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:06.967429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:07.048716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:07.087709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a5af1-038a-4028-b4f9-6264c438b878 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:07.099530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79703b8-1afe-4547-8afb-0ba2a689f903 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:07.140079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:07.140842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:07.141193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:07.141536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] PATH_INFO: `/v3/users/e060a7db58524858b7c229656d25097a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:07.147044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] RBAC: Authorizing `identity:get_user(user_id=e060a7db58524858b7c229656d25097a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:07.148673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:07.154360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 129/261] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:06 2022] GET /identity/v3/users/e060a7db58524858b7c229656d25097a => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:07.156085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:07.159066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:07.159912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:07.160388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:07.160853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:07.161722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:07.162262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:07.163966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:07.178387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 133/262] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:07.179615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:07.202366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:07.308391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624eeb3-50d7-445e-b56b-4b0396f80df6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:07.345782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:07.346803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:07.347285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:07.347774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] PATH_INFO: `/v3/users/e060a7db58524858b7c229656d25097a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:07.349160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] RBAC: Authorizing `identity:get_user(user_id=e060a7db58524858b7c229656d25097a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:07.350861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:07.352702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 134/263] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:07 2022] GET /identity/v3/users/e060a7db58524858b7c229656d25097a => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:07.359267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:07.401281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e41e5-334b-42b0-a89f-45646828e630 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:07.442972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:07.444718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:07.466924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7d66-f3ac-4b71-9462-70cb4f919a50 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:07.467120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:07.467746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:07.468214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:07.468489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:07.504689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:07.505313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:07.505597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:07.505870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:07.595217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:07.596706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:07.615389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:07.616878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:07.763304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 130/264] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:14:07.773482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:07.780849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:07.820965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:07.827097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 135/265] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:14:07.833360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:07.970798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5adf9b-8cad-4588-a214-9483cd2b8b19 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:07.979465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01e28fa-cb62-4ca2-a437-c9b8529afd42 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:08.009813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e817313-3171-4ae3-a47d-980d5e2b66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.011789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e817313-3171-4ae3-a47d-980d5e2b66b2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.013437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e817313-3171-4ae3-a47d-980d5e2b66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.014114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e817313-3171-4ae3-a47d-980d5e2b66b2 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.017677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e817313-3171-4ae3-a47d-980d5e2b66b2 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:08.022280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e817313-3171-4ae3-a47d-980d5e2b66b2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:08.024902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 136/266] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:07 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:08.027877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3303d00-4614-4050-8519-b25a9aaf831d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.028769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3303d00-4614-4050-8519-b25a9aaf831d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.031404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3303d00-4614-4050-8519-b25a9aaf831d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.034129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 137/267] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:08.036113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3303d00-4614-4050-8519-b25a9aaf831d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:08.054839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31587a9-e5cf-4a38-9b5e-5cf10df36c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.057675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31587a9-e5cf-4a38-9b5e-5cf10df36c7e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.058170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31587a9-e5cf-4a38-9b5e-5cf10df36c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.058724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31587a9-e5cf-4a38-9b5e-5cf10df36c7e None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.061808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31587a9-e5cf-4a38-9b5e-5cf10df36c7e None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:08.063541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31587a9-e5cf-4a38-9b5e-5cf10df36c7e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:08.068055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 131/268] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:07 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:08.069937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be2f34-4d4c-41d1-b56e-6cb4f431d2e4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.070230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be2f34-4d4c-41d1-b56e-6cb4f431d2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.074667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be2f34-4d4c-41d1-b56e-6cb4f431d2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.094967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-26be2f34-4d4c-41d1-b56e-6cb4f431d2e4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:08.096625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-26be2f34-4d4c-41d1-b56e-6cb4f431d2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:08.114201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:08.179633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3303d00-4614-4050-8519-b25a9aaf831d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:08.180989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 132/269] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:08.183573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee52a7d-8b50-4ca3-9956-c8c0971abb77 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.184116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee52a7d-8b50-4ca3-9956-c8c0971abb77 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.184612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee52a7d-8b50-4ca3-9956-c8c0971abb77 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.186090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 133/270] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:08.187030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee52a7d-8b50-4ca3-9956-c8c0971abb77 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:08.224804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:08.258630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.259638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.260700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.261369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.263612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] RBAC: Adding query filter params (role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=e060a7db58524858b7c229656d25097a) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:08.264989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:08.266700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260f5fb-d903-45ea-ab48-1853d4208be5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:08.281591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 138/271] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:08 2022] GET /identity/v3/role_assignments?user.id=e060a7db58524858b7c229656d25097a&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=3a297fae4df9419b9c53354550781a57 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:08.283393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.285207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.285749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.320719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:08.326222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:08.408947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 139/272] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:08.428115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee52a7d-8b50-4ca3-9956-c8c0971abb77 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:08.485090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.485632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.485910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.486176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.486957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=e060a7db58524858b7c229656d25097a) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:08.487259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:08.488764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec2798-3a73-429f-a9a4-ca294927f4ab None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:08.499882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 134/273] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:08 2022] GET /identity/v3/role_assignments?user.id=e060a7db58524858b7c229656d25097a&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:08.534474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:08.562358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe399a5-ba2e-48bb-9622-e01141394011 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.562358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe399a5-ba2e-48bb-9622-e01141394011 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.562769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe399a5-ba2e-48bb-9622-e01141394011 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.564057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 135/274] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:08.570525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.570814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.571016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.594723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:08.611465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:08.612692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:08.659596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4614a-4d45-45ac-aa13-cf4cd2078447 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:08.698074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.698842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.699130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.699505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.711402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:08.717024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:08.717911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:08.718733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 140/275] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:08.729461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:08.775081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 136/276] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:08.786929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:08.813952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:08.874801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:08.880573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfcbc0-2951-4005-a044-0e4336b51166 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:08.921321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.922117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.922528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.922901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.923685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:08.924084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:08.925690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:08.935944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 141/277] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:08.941451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:08.942187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2272ecf-4920-40b4-9710-76b99aa6fbd6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:08.991167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:08.992498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:08.992925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:08.993374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:08.995226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:08.997574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:09.000207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 137/278] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:09.005748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:09.060798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d71d5-53c9-4bc2-8aa0-5fb87bb69610 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:09.096846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:09.097572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.097925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.098269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] PATH_INFO: `/v3/users/7dd8f075cf8a4571950a40e46048f7b2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.112727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd8f075cf8a4571950a40e46048f7b2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:09.114287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:09.116540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:09.121903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 142/279] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:08 2022] GET /identity/v3/users/7dd8f075cf8a4571950a40e46048f7b2 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:09.123451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:09.136909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3914fa7-be9f-498a-849a-b121a476e17e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:09.154511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:09.175997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:09.176574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.176855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.177126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.177788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:09.179286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:09.245421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300efc8-85de-457a-a56f-7444a6805468 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:09.275846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 138/280] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:09 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:09.276745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:09.296607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:09.297238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.297520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.297798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.298573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, user.id=7dd8f075cf8a4571950a40e46048f7b2, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:09.298890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:09.300358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:09.322932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 143/281] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Sep 3 01:14:09 2022] GET /identity/v3/role_assignments?user.id=7dd8f075cf8a4571950a40e46048f7b2&scope.system=all&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:09.408565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fd6fb-40d7-4153-a416-da498a52b3c9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:09.451296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c170cf3-0f60-478e-aca5-080f39858954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:09.452263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c170cf3-0f60-478e-aca5-080f39858954 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.452770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c170cf3-0f60-478e-aca5-080f39858954 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.453210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c170cf3-0f60-478e-aca5-080f39858954 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.462098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:09.468837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c170cf3-0f60-478e-aca5-080f39858954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6c170cf3-0f60-478e-aca5-080f39858954 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:09.476288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:09.480059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:09.481321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 139/282] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:09.488093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:09.589648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:09.606264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05a734-6d05-4c8e-8ca6-f00e49f3c295 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:09.649507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:09.650123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.650430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.650761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.651430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:09.651778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:09.653345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73c902-beb3-4fb6-8925-abb58c7f19dd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:09.659151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbcf4b-3cfd-42b2-a4dd-45dab44d3928 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.659432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbcf4b-3cfd-42b2-a4dd-45dab44d3928 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.659701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbcf4b-3cfd-42b2-a4dd-45dab44d3928 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.663682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 144/283] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:09.665708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 140/284] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:09.666802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.667289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.667756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.669724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbcf4b-3cfd-42b2-a4dd-45dab44d3928 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:09.691422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:09.692847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:09.778045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 145/285] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:09.810718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbcf4b-3cfd-42b2-a4dd-45dab44d3928 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:09.851330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:09.851891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:09.852170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:09.852441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] PATH_INFO: `/v3/users/480d632482144cdfb663ba1a25be8c75` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:09.853571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] RBAC: Authorizing `identity:get_user(user_id=480d632482144cdfb663ba1a25be8c75)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:09.855087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:09.857573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 141/286] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:09 2022] GET /identity/v3/users/480d632482144cdfb663ba1a25be8c75 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:09.863184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:09.980912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:10.010296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf9eb-ce42-4c50-aa6e-b3d5c642a0b4 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:10.050204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:10.051040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:10.051413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:10.051778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:10.052989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:10.054555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:10.056581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 146/287] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:09 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:10.062170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:10.111459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:10.144276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:10.205377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f3f54-3c33-4886-8357-5c4f1443ab14 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:10.224598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1104e-e9bb-4d64-a23b-ffd7720dcab4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:10.266528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:10.266528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:10.266528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:10.266528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889/users/480d632482144cdfb663ba1a25be8c75/roles/a40cb285917842ecb5be4fd4b47cc319` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:10.266938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] RBAC: Authorizing `identity:create_grant(project_id=314c6a4b7a99410c99ac353aff468889, user_id=480d632482144cdfb663ba1a25be8c75, role_id=a40cb285917842ecb5be4fd4b47cc319)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:10.266938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:10.275107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:10.275752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:10.276031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:10.276301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:10.287310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:10.293341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:10.294249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:10.295100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 147/288] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:10.303396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:10.308423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:10.346874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 142/289] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:10 2022] PUT /identity/v3/projects/314c6a4b7a99410c99ac353aff468889/users/480d632482144cdfb663ba1a25be8c75/roles/a40cb285917842ecb5be4fd4b47cc319 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:10.421397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859e0d8-db54-4a14-a6cf-a0861564a8d9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:10.461380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:10.462416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:10.462874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:10.463320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:10.464268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:10.464779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:10.467175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:10.477480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 148/290] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:10.487513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:10.628571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989d030-7d83-4aed-b0fb-dbf31df7ce1c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:10.656620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:10.665757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:10.666469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:10.666773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:10.667087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] PATH_INFO: `/v3/users/8f50fcdc926e40dc89c2f2fec4c37ea6` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:10.677682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] RBAC: Authorizing `identity:get_user(user_id=8f50fcdc926e40dc89c2f2fec4c37ea6)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:10.677682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:10.677682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 143/291] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:10 2022] GET /identity/v3/users/8f50fcdc926e40dc89c2f2fec4c37ea6 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:10.677931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:10.761070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:10.813053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713218f-3336-4d90-b357-796f0dd080f3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:10.858274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:10.858917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:10.859193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:10.859462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:10.953137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:10.954567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:11.037304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:11.084395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 149/292] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:11.089554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:11.232590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a830014-ac82-4b40-b288-ec9187ee1129 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:11.269395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4d115-adeb-4f51-89da-1dbfc6050bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:11.270284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4d115-adeb-4f51-89da-1dbfc6050bbf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:11.270606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4d115-adeb-4f51-89da-1dbfc6050bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:11.270874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4d115-adeb-4f51-89da-1dbfc6050bbf None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:11.271981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4d115-adeb-4f51-89da-1dbfc6050bbf None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:11.273423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4d115-adeb-4f51-89da-1dbfc6050bbf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:11.275378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 144/293] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:11 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:11.281166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:11.424147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616d99-50c5-45c4-ba74-67e65b6cddb4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:11.472770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:11.473390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:11.473672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:11.473945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:11.474708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=8f50fcdc926e40dc89c2f2fec4c37ea6) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:11.475016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:11.476455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd995a9f-6163-4ffc-90db-fcc2447d308c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:11.482710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 150/294] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:11 2022] GET /identity/v3/role_assignments?user.id=8f50fcdc926e40dc89c2f2fec4c37ea6&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:11.597200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66119aaf-e0ff-4137-896e-c8fa05614292 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:11.597865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66119aaf-e0ff-4137-896e-c8fa05614292 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:11.598365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66119aaf-e0ff-4137-896e-c8fa05614292 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:11.600469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 145/295] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:11.607061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cfcef-8354-4cef-98c3-4a9c091de778 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:11.607353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cfcef-8354-4cef-98c3-4a9c091de778 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:11.607612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8cfcef-8354-4cef-98c3-4a9c091de778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:11.631478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4f8cfcef-8354-4cef-98c3-4a9c091de778 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:11.632703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f8cfcef-8354-4cef-98c3-4a9c091de778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:11.661728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:11.707286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 151/296] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:11.893065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a580f-83fc-4263-9a94-360309602f6f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:11.893411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a580f-83fc-4263-9a94-360309602f6f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:11.893632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a580f-83fc-4263-9a94-360309602f6f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:11.895528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 146/297] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:11.900880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:11.901175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:11.901453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:11.925097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:11.926310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:12.010816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 152/298] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:12.021807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a580f-83fc-4263-9a94-360309602f6f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.101547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.143402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a580f-83fc-4263-9a94-360309602f6f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.180565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f6c65-d5aa-4465-9ec3-e9217c959b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.181202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f6c65-d5aa-4465-9ec3-e9217c959b5d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.181482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f6c65-d5aa-4465-9ec3-e9217c959b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.181753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f6c65-d5aa-4465-9ec3-e9217c959b5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.192345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f6c65-d5aa-4465-9ec3-e9217c959b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-756f6c65-d5aa-4465-9ec3-e9217c959b5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:12.197609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:12.198507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:12.199864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 147/299] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:12.206479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.216415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6ed64-88a4-4158-82d6-487785fffea4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.216702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6ed64-88a4-4158-82d6-487785fffea4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.216979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6ed64-88a4-4158-82d6-487785fffea4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.222193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 148/300] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:14:12.223346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.223636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.223917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.242554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:12.243769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:12.316091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.316278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 149/301] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:12.319230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.326951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3a575-0e60-46f9-9ab0-0371da6582c7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.370270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.370835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.371661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.371990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.372661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:12.372965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:12.374927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:12.381265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.384884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 153/302] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:12.390499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.435676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132e3af-b47a-47de-a84c-94c406acc584 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.474558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.475556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.476095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.476608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.488500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:12.493963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:12.494870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:12.495644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:12.503450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b43db2-21a4-4b3e-bcb2-7bca205cab91 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.504010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 150/303] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:12.505067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.535126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.535759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.536042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.536310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] PATH_INFO: `/v3/users/7dd8f075cf8a4571950a40e46048f7b2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.537449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd8f075cf8a4571950a40e46048f7b2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:12.539035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:12.540912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 154/304] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/users/7dd8f075cf8a4571950a40e46048f7b2 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:12.546802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.571259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.627895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dc521-d868-4488-8bcf-c95204f16006 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.641925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.656072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c53fae-0d56-4d03-b55d-e6acabb3f373 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.665356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.666077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.666384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.666661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.667332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:12.667634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:12.669710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:12.682382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 151/305] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:12.682382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.697624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.698357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.698615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.698891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] PATH_INFO: `/v3/system/users/7dd8f075cf8a4571950a40e46048f7b2/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.700400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7dd8f075cf8a4571950a40e46048f7b2, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:12.701792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:12.766203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 155/306] 10.222.0.13 () {58 vars in 1334 bytes} [Sat Sep 3 01:14:12 2022] PUT /identity/v3/system/users/7dd8f075cf8a4571950a40e46048f7b2/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:12.767843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.816602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806b53-fc41-4b27-a787-294735f5323b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.862469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ff58b-5d32-4b71-be5c-8bee6b0f0ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.863588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff58b-5d32-4b71-be5c-8bee6b0f0ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.864068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff58b-5d32-4b71-be5c-8bee6b0f0ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.864529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff58b-5d32-4b71-be5c-8bee6b0f0ab0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.865956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff58b-5d32-4b71-be5c-8bee6b0f0ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:12.867698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff58b-5d32-4b71-be5c-8bee6b0f0ab0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:12.868776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.869927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 152/307] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:12.871338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd3764-2976-4f04-b89c-8a85885aca25 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.871683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd3764-2976-4f04-b89c-8a85885aca25 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.872028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd3764-2976-4f04-b89c-8a85885aca25 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.873597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 153/308] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:12.875313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd3764-2976-4f04-b89c-8a85885aca25 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:12.891484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d8654-e029-4014-a736-60bc971b556a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:12.930762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:12.936426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a4a90-a744-46f8-b894-e175f896d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:12.937071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a4a90-a744-46f8-b894-e175f896d0ff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.937355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a4a90-a744-46f8-b894-e175f896d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.937631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a4a90-a744-46f8-b894-e175f896d0ff None admin] PATH_INFO: `/v3/users/af5a2d9bb3244b268e23e560bea3d04d` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.953354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a4a90-a744-46f8-b894-e175f896d0ff None admin] RBAC: Authorizing `identity:get_user(user_id=af5a2d9bb3244b268e23e560bea3d04d)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:12.954865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a4a90-a744-46f8-b894-e175f896d0ff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:12.956852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 156/309] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:12 2022] GET /identity/v3/users/af5a2d9bb3244b268e23e560bea3d04d => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:12.957684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:12.957962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:12.958226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:12.977048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:12.978228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:13.001299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd3764-2976-4f04-b89c-8a85885aca25 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:13.039444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:13.040089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:13.040370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:13.040644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] PATH_INFO: `/v3/groups` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:13.041279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:13.043271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:13.050535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 157/310] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:13.051375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:13.113057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 154/311] 10.222.0.13 () {60 vars in 1134 bytes} [Sat Sep 3 01:14:12 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:13.129618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:13.180094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b01ae-0f2d-4430-9a4d-340ee1108266 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:13.219658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:13.221081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:13.221556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:13.222040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:13.223018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] RBAC: Adding query filter params (role.id=a40cb285917842ecb5be4fd4b47cc319, user.id=af5a2d9bb3244b268e23e560bea3d04d, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:13.223522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:13.225314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:13.232679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 158/312] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Sep 3 01:14:13 2022] GET /identity/v3/role_assignments?user.id=af5a2d9bb3244b268e23e560bea3d04d&scope.system=all&role.id=a40cb285917842ecb5be4fd4b47cc319 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:13.349252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:13.460186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b64b1b-0691-453e-97ab-4ebde472fb23 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:13.462244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:13.501375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:13.502143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:13.502456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:13.502730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:13.519286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:13.524493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:13.526635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:13.527535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 155/313] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:13.531863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:13.637209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fac639-01c4-4cfb-bc29-c60a59ee9b8b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:13.672726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:13.673338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:13.673619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:13.673887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:13.674577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:13.674880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:13.676260 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:13.678572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:13.685917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 159/314] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:13.691330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:13.806963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfd248-c85f-47d9-bc19-aab12f7261af None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:13.848190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:13.848820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:13.849100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:13.849370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] PATH_INFO: `/v3/users/480d632482144cdfb663ba1a25be8c75` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:13.850525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] RBAC: Authorizing `identity:get_user(user_id=480d632482144cdfb663ba1a25be8c75)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:13.851980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:13.853863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 156/315] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:13 2022] GET /identity/v3/users/480d632482144cdfb663ba1a25be8c75 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:13.860142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:13.967722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f059f-6d5f-4c92-83e0-1093d41727bd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:14.003197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:14.004026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:14.004481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:14.004941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:14.074291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:14.077891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:14.079473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:14.180706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 160/316] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:14.186595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:14.221331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:14.297940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a971a0-11b0-4a6b-81c6-db6b76ec3375 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:14.332491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83139399-35f9-4ffb-b6a4-adca7bc6aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:14.333045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83139399-35f9-4ffb-b6a4-adca7bc6aab0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:14.333321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83139399-35f9-4ffb-b6a4-adca7bc6aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:14.333586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83139399-35f9-4ffb-b6a4-adca7bc6aab0 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:14.334723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83139399-35f9-4ffb-b6a4-adca7bc6aab0 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:14.336181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83139399-35f9-4ffb-b6a4-adca7bc6aab0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:14.338110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 157/317] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:14 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:14.344065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:14.443452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:14.447202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1553d-0bd7-4a8f-b1e0-d6df0a25eb77 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:14.484850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:14.485670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:14.486012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:14.486461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:14.487252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] RBAC: Adding query filter params (role.id=a40cb285917842ecb5be4fd4b47cc319, scope.project.id=314c6a4b7a99410c99ac353aff468889, user.id=480d632482144cdfb663ba1a25be8c75) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:14.487881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:14.489320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9924f-f1b6-4c6f-9f60-2522ea40434b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:14.495579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 161/318] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:14 2022] GET /identity/v3/role_assignments?user.id=480d632482144cdfb663ba1a25be8c75&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=a40cb285917842ecb5be4fd4b47cc319 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:14.831350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63a287-07a1-4ca4-b8c8-3a1eab02f5fc None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:14.831730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63a287-07a1-4ca4-b8c8-3a1eab02f5fc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:14.832102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63a287-07a1-4ca4-b8c8-3a1eab02f5fc None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:14.833696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 158/319] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:14.838611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d975c3-3db6-4b0b-9f02-8c6b5cf54912 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:14.838905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d975c3-3db6-4b0b-9f02-8c6b5cf54912 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:14.839183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d975c3-3db6-4b0b-9f02-8c6b5cf54912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:14.858156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-94d975c3-3db6-4b0b-9f02-8c6b5cf54912 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:14.859404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-94d975c3-3db6-4b0b-9f02-8c6b5cf54912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:14.930326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 162/320] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:15.124702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b9451-038c-45f4-82b3-ec46e0e31cab None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.125045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b9451-038c-45f4-82b3-ec46e0e31cab None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.125250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b9451-038c-45f4-82b3-ec46e0e31cab None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.126702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 159/321] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:15.131740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b104d-c23a-494c-b751-bd3b8a314918 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.132027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b104d-c23a-494c-b751-bd3b8a314918 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.132299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b104d-c23a-494c-b751-bd3b8a314918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.150808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-655b104d-c23a-494c-b751-bd3b8a314918 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:15.151999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-655b104d-c23a-494c-b751-bd3b8a314918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:15.201580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b9451-038c-45f4-82b3-ec46e0e31cab None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.220896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 163/322] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:15.298090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8858ed-7f6e-4ac8-a06f-79ccc81c375c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.298399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8858ed-7f6e-4ac8-a06f-79ccc81c375c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.298673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8858ed-7f6e-4ac8-a06f-79ccc81c375c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.300134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 164/323] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:14:15.304737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870759-0331-4948-8915-1f22696dd476 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.305016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870759-0331-4948-8915-1f22696dd476 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.305283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38870759-0331-4948-8915-1f22696dd476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.305854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:15.310827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b9451-038c-45f4-82b3-ec46e0e31cab None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.323434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-38870759-0331-4948-8915-1f22696dd476 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:15.324712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-38870759-0331-4948-8915-1f22696dd476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:15.342438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:15.346356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.347009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.347292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.347625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.359101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:15.364031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:15.364940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:15.365741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:15.370039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 160/324] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:15.371366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.394909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 165/325] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:15.476628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8e2e4-9f98-42d8-996a-e2e8cdf24064 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.493796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870759-0331-4948-8915-1f22696dd476 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.514912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.515523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.515805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.516073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.516735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:15.517034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:15.518450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:15.527108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 161/326] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:15.532424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.554152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:15.619620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38870759-0331-4948-8915-1f22696dd476 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.640254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aad7b1-1c82-41bb-a171-cc83bfa027a9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.659244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.659875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.660078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.660344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.661443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:15.663876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:15.667187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 166/327] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:15.668558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.681863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.682535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.682816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.683088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] PATH_INFO: `/v3/users/7dd8f075cf8a4571950a40e46048f7b2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.684544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd8f075cf8a4571950a40e46048f7b2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:15.685994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:15.687657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 162/328] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/users/7dd8f075cf8a4571950a40e46048f7b2 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:15.688349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.694752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:15.773668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711477c0-5612-45c8-bbd1-d47b16462d57 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.775180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:15.787427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54595-41fd-4deb-8dbb-37cae7444430 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.807802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.808478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.808816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.809092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.819385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:15.823774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:15.824796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:15.825642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:15.826472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:15.827107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.827107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.827107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] PATH_INFO: `/v3/groups` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.827290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 167/329] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:15.827365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:15.827443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.828174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:15.907478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:15.926381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ca4df-b809-4679-a997-6b207c469e53 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:15.930996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 163/330] 10.222.0.13 () {60 vars in 1134 bytes} [Sat Sep 3 01:14:15 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:15.931849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:15.966617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:15.967255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:15.967536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:15.967810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:15.968731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] RBAC: Adding query filter params (role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, user.id=7dd8f075cf8a4571950a40e46048f7b2, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:15.969048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:15.970556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:15.998423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 168/331] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/role_assignments?user.id=7dd8f075cf8a4571950a40e46048f7b2&scope.system=all&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:16.059874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1e560-a0ec-416f-b3f7-971cbb8f536a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:16.098858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:16.099603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:16.099954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:16.100305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:16.101089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:16.101464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:16.102946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:16.113817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 164/332] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:16.119623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:16.234016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28d283-cf34-41b7-8dd0-6364bf8e07b8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:16.270465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15027fe-e763-4cfd-946c-d0d900848cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:16.271106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15027fe-e763-4cfd-946c-d0d900848cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:16.271387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15027fe-e763-4cfd-946c-d0d900848cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:16.271658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15027fe-e763-4cfd-946c-d0d900848cc7 None admin] PATH_INFO: `/v3/users/af5a2d9bb3244b268e23e560bea3d04d` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:16.272794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15027fe-e763-4cfd-946c-d0d900848cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=af5a2d9bb3244b268e23e560bea3d04d)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:16.274261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15027fe-e763-4cfd-946c-d0d900848cc7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:16.276034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 169/333] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:16 2022] GET /identity/v3/users/af5a2d9bb3244b268e23e560bea3d04d => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:16.281819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:16.295351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:16.394724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7aa28-e098-429b-a93d-a8821257680f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:16.432003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671c51-2835-489c-bbbf-5c210df3091c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:16.432663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671c51-2835-489c-bbbf-5c210df3091c None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:16.432944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671c51-2835-489c-bbbf-5c210df3091c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:16.433220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671c51-2835-489c-bbbf-5c210df3091c None admin] PATH_INFO: `/v3/system/users/af5a2d9bb3244b268e23e560bea3d04d/roles/a40cb285917842ecb5be4fd4b47cc319` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:16.434893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43671c51-2835-489c-bbbf-5c210df3091c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=af5a2d9bb3244b268e23e560bea3d04d, role_id=a40cb285917842ecb5be4fd4b47cc319)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:16.436309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43671c51-2835-489c-bbbf-5c210df3091c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:16.515310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 165/334] 10.222.0.13 () {58 vars in 1334 bytes} [Sat Sep 3 01:14:16 2022] PUT /identity/v3/system/users/af5a2d9bb3244b268e23e560bea3d04d/roles/a40cb285917842ecb5be4fd4b47cc319 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:18.432510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f38193-7b50-4034-8c92-f3e00a466ef1 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:18.434119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f38193-7b50-4034-8c92-f3e00a466ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:18.434570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f38193-7b50-4034-8c92-f3e00a466ef1 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:18.436276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 170/335] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:18.444709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459b6b5-de58-4a7b-aad1-31d57e94057f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:18.445590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459b6b5-de58-4a7b-aad1-31d57e94057f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:18.446075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459b6b5-de58-4a7b-aad1-31d57e94057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:18.471351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f459b6b5-de58-4a7b-aad1-31d57e94057f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:18.472880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f459b6b5-de58-4a7b-aad1-31d57e94057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:18.555054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 166/336] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:18.578907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b8af-18c3-4f5d-a617-acdb82388c76 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:18.579588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b8af-18c3-4f5d-a617-acdb82388c76 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:18.580056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804b8af-18c3-4f5d-a617-acdb82388c76 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:18.581598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 171/337] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:18.586827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:18.587113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:18.587381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:18.599322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:18.605822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4294-21b3-45ad-af9a-1d926d95ef8c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:18.606421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4294-21b3-45ad-af9a-1d926d95ef8c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:18.606925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a4294-21b3-45ad-af9a-1d926d95ef8c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:18.608744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 172/338] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:18.611886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:18.618994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:18.619105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:18.619105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:18.619105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:18.638378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:18.647358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:18.653337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:18.739336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 167/339] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:18.788959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 173/340] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:18.941280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.063783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8492ec-841d-4dd5-8ccb-e5f0a34da09d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.108307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.108965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.109243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.109511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.116015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:19.128688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:19.133666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:19.134650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:19.135443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 168/341] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:19.142590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.221738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.271037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f9cfa-3eb5-404e-8152-648406762338 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.312330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.313323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.313982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.315255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.316329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:19.319453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:19.321062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:19.326096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:19.338468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 174/342] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:19.345151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.429471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170168c-6989-4679-bf05-274845d6e29d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.468579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.469227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.469492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.469768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.481287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:19.486658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:19.487529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:19.488521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:19.490609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775323-2880-40a1-9f49-87154b23d459 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.491346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 169/343] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:19.492302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.534766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.535565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.536062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.536509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.558746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:19.607691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4304bb4c-ee1b-43e2-9a16-0c4ce359ee3f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.608419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:19.612759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:19.615520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:19.654117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafac3b-9837-4d43-bdb3-0aa16ded4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.655102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafac3b-9837-4d43-bdb3-0aa16ded4da5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.655589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafac3b-9837-4d43-bdb3-0aa16ded4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.656087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafac3b-9837-4d43-bdb3-0aa16ded4da5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.667224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafac3b-9837-4d43-bdb3-0aa16ded4da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-acafac3b-9837-4d43-bdb3-0aa16ded4da5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:19.673632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:19.674564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:19.675451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:19.677733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 170/344] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:19.679367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5cd3e-5df2-487b-99dd-df6e5716526b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.679866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5cd3e-5df2-487b-99dd-df6e5716526b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.680346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5cd3e-5df2-487b-99dd-df6e5716526b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.685402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 171/345] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:19.686443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5cd3e-5df2-487b-99dd-df6e5716526b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.727879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 175/346] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:19.728728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:19.809858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5cd3e-5df2-487b-99dd-df6e5716526b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.829911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:19.840053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:19.845592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bd36d-d7c3-4588-b3a1-db9a18d9d3f6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:19.859676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.859676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.859676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.859996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.860498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:19.860810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:19.862197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e2779-0eac-4646-94bd-374955ccaf9e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:19.879057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 172/347] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:19.880589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.881011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.881438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.900650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:19.901643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:19.902680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:19.903155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:19.904075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:19.904583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:19.906269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:19.908413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:19.910409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:19.918912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 176/348] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:19.920984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.005870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 173/349] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:20.006822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.062418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb429378-5b2f-4135-a6cc-d2519245f6c4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.101948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.102614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.103363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.103646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.105676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:20.107997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:20.110738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 177/350] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:19 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:20.111488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.124025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5faa23-6593-420d-afac-2f5c2d3d5277 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.132308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.150550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.160599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a2f28-dcab-4566-97a5-6dffb52ff16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.161211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2f28-dcab-4566-97a5-6dffb52ff16c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.161523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2f28-dcab-4566-97a5-6dffb52ff16c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.161760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a2f28-dcab-4566-97a5-6dffb52ff16c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.239308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2f28-dcab-4566-97a5-6dffb52ff16c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:20.240895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14a2f28-dcab-4566-97a5-6dffb52ff16c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:20.248507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3efaf6c-feca-44d5-8980-a1089d19cf10 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.293512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.294422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.294705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.295033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.361055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 174/351] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:20.362329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b89976-ba52-4d37-83c1-055d5a1d6b53 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.362670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b89976-ba52-4d37-83c1-055d5a1d6b53 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.362943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b89976-ba52-4d37-83c1-055d5a1d6b53 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.364416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 175/352] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:20.365120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b89976-ba52-4d37-83c1-055d5a1d6b53 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.376790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.390312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:20.391814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:20.426972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.501812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b89976-ba52-4d37-83c1-055d5a1d6b53 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.518390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 178/353] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:20.519223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.539854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac9f52-cc32-4808-bc8f-439fabe3132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.540489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9f52-cc32-4808-bc8f-439fabe3132f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.540764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9f52-cc32-4808-bc8f-439fabe3132f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.541031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac9f52-cc32-4808-bc8f-439fabe3132f None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.547862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac9f52-cc32-4808-bc8f-439fabe3132f None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:20.549472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac9f52-cc32-4808-bc8f-439fabe3132f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:20.551540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 176/354] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:20.552502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.552781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.553166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.575855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:20.577079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:20.648219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d19e27-a3b0-404d-a5ff-12912546653b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.651531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.656646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 177/355] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:20.657601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.682998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.693902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb408d8-dfb1-4d90-b59d-526987a79732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.694736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb408d8-dfb1-4d90-b59d-526987a79732 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.695086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb408d8-dfb1-4d90-b59d-526987a79732 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.695432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb408d8-dfb1-4d90-b59d-526987a79732 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.696697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb408d8-dfb1-4d90-b59d-526987a79732 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:20.698280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb408d8-dfb1-4d90-b59d-526987a79732 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:20.701204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 179/356] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:20.704149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6444bb8-c128-47df-8425-ad38a8acdc59 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.704149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6444bb8-c128-47df-8425-ad38a8acdc59 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.704149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6444bb8-c128-47df-8425-ad38a8acdc59 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.704593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 180/357] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:20.705063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6444bb8-c128-47df-8425-ad38a8acdc59 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.775253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37253e-6e31-42d8-8dc0-e17b0917cd03 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.821266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.821907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.822465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.822739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.834645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6444bb8-c128-47df-8425-ad38a8acdc59 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.838039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:20.843930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:20.844830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:20.845603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 178/358] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:20.851334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:20.879885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc051-14fa-4d7e-a188-f4c979333fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:20.880519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc051-14fa-4d7e-a188-f4c979333fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.880801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc051-14fa-4d7e-a188-f4c979333fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.881069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc051-14fa-4d7e-a188-f4c979333fe5 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.882990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc051-14fa-4d7e-a188-f4c979333fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:20.884696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc051-14fa-4d7e-a188-f4c979333fe5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:20.886579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 181/359] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:20.887797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:20.888080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:20.888347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:20.914013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:20.915323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:20.927509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.973288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fb48f-ea3f-4424-9f30-7c0ee5081986 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:20.987047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:20.996576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 182/360] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:20.997425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.013362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.013974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.014253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.014589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.015322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=591bfc2823974e3f9466598de018ddce) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:21.015819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.017428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.023480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 179/361] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=3a297fae4df9419b9c53354550781a57 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.024355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.138605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42926ce4-958b-4cb5-a3f9-6db25ebf32ba None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.169563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30277a4b-9e8e-42df-a643-ec877d96b28b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.196425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.197075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.197349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.197613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.199441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.201189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.203093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 183/362] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:20 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.204315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:21.204425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.215889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.216817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.220752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.220752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.220752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:21.220752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.220752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.235870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 180/363] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.236978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.284841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:21.333137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b6365-5f1c-4623-89a4-217c6f905078 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.351095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1dead-38d5-4d47-a0b7-10b8549beace None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.379613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.380262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.380542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.380813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.382431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.383900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.385664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 184/364] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.386364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.395548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.396318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.396570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.396839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.410388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:21.415353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:21.416253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:21.417047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 181/365] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:21.417809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.479609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:21.522387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e7757-03cf-42a7-b84c-9142136a63b5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.552521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f38454-1936-4827-a4c9-02b0e51799ab None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.560918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:21.571306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.571503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.571710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.571980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.572988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=314c6a4b7a99410c99ac353aff468889) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:21.573838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.576694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.584146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 185/366] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.585145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.598621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.599668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.600156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.600639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] PATH_INFO: `/v3/users/af5a2d9bb3244b268e23e560bea3d04d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.602185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] RBAC: Authorizing `identity:get_user(user_id=af5a2d9bb3244b268e23e560bea3d04d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.604097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.606735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 182/367] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/users/af5a2d9bb3244b268e23e560bea3d04d => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.607829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.721081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976ad8-4be9-4f0c-86d1-53f316b42a80 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.721299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf479e-554f-4166-b5ea-ecac7b36e5f1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:21.738671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:21.760577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.762131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.762539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.762913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.764036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=591bfc2823974e3f9466598de018ddce) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:21.764352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:21.764957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:21.765619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:21.765869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:21.765983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:21.766153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:21.773098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 183/368] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:21.773938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.779270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:21.784784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:21.785656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:21.786582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 186/369] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:21.787249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:21.900052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:21.991607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7db315-f385-48ef-a15f-023124db4268 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.005047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0ada3-c74f-4cad-8ec7-56a2e5998a23 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.034185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.034904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.035194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.035529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.036209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:22.036563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.038008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.048084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 184/370] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:22.048914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.058780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.059554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.059953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.060326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.061168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] RBAC: Adding query filter params (role.id=a40cb285917842ecb5be4fd4b47cc319, user.id=af5a2d9bb3244b268e23e560bea3d04d, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:22.061753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.063731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.091670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 187/371] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Sep 3 01:14:21 2022] GET /identity/v3/role_assignments?user.id=af5a2d9bb3244b268e23e560bea3d04d&scope.system=all&role.id=a40cb285917842ecb5be4fd4b47cc319 => generated 510 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:22.092556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.134717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:22.208711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7df8-c45e-4bdd-9a85-21f6f9376a58 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.219571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d6ec1-e318-48ef-9bd3-14d892bc1aa8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.234781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:22.260846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.261517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.261796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.262083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.266531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.268181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.268478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.268877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.269575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:22.269881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.271835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.287421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 185/372] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:22.293339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.344114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.345625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.413463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ceaba-bc20-4c97-b677-7c05a0fac85f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.438698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:22.454013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.454759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.455042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.455418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.465288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 188/373] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:22.471284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.487396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:22.535433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.537040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.587629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8976851-ba80-42bc-b5fe-318860107ea5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.627315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.627962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.628245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.628524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.632096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.633691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.638702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 189/374] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:22.647579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.713893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 186/375] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:14:22.719483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.722191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:22.817580 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:22.860997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d3bd5-e762-4c4a-991f-8f0c069edbd5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.906458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.907105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.907390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.907669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.908806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.910272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.911905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 190/376] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:22.919339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:22.921770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b09e6-562b-490c-825d-3cdb5e5f93d8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:22.973007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:22.973654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:22.973930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:22.974195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] PATH_INFO: `/v3/projects/06b321cae6a24e3a9217594a16dab365` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:22.976027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] RBAC: Authorizing `identity:get_project(project_id=06b321cae6a24e3a9217594a16dab365)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:22.977481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:22.979141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 187/377] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/projects/06b321cae6a24e3a9217594a16dab365 => generated 344 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:22.985258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:23.078398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:23.088427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7652422-1871-44f0-afde-40b53a79673e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:23.114258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532385b-6be8-48a6-847b-e2296b20d8f4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:23.139151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:23.139766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:23.140043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:23.140890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:23.141777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=314c6a4b7a99410c99ac353aff468889) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:23.142218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:23.144685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:23.153133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 191/378] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=3a297fae4df9419b9c53354550781a57 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:23.162058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a59c59-fd5f-4463-8532-2ac09768d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:23.162756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a59c59-fd5f-4463-8532-2ac09768d780 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:23.163035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a59c59-fd5f-4463-8532-2ac09768d780 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:23.163307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a59c59-fd5f-4463-8532-2ac09768d780 None admin] PATH_INFO: `/v3/groups/cb778f9298a648b3b9b0d2a605192e9b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:23.170085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a59c59-fd5f-4463-8532-2ac09768d780 None admin] RBAC: Authorizing `identity:get_group(group_id=cb778f9298a648b3b9b0d2a605192e9b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:23.171748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a59c59-fd5f-4463-8532-2ac09768d780 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:23.173759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 188/379] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:22 2022] GET /identity/v3/groups/cb778f9298a648b3b9b0d2a605192e9b => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:23.179914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:23.197038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:23.315934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688da567-6c25-45a5-ac91-87fb68ac8d2d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:23.354778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:23.356213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:23.356684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:23.357132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:23.358056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] RBAC: Adding query filter params (group.id=cb778f9298a648b3b9b0d2a605192e9b, role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=06b321cae6a24e3a9217594a16dab365) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:23.358563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:23.360249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1a860-ba03-4dc4-9e52-3d110bd03648 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:23.366758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 192/380] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:23 2022] GET /identity/v3/role_assignments?group.id=cb778f9298a648b3b9b0d2a605192e9b&scope.project.id=06b321cae6a24e3a9217594a16dab365&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:23.517016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217c614-e371-45b6-bf59-e71223eb9a2e None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:23.517299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217c614-e371-45b6-bf59-e71223eb9a2e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:23.517576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217c614-e371-45b6-bf59-e71223eb9a2e None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:23.519287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 189/381] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:23.524773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:23.525359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:23.525858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:23.555571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:23.557170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:23.636517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 193/382] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:24.010332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217c614-e371-45b6-bf59-e71223eb9a2e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:24.139787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217c614-e371-45b6-bf59-e71223eb9a2e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:24.178578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4adbac-7703-4538-917b-50359a0bc416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:24.179232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adbac-7703-4538-917b-50359a0bc416 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.179603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adbac-7703-4538-917b-50359a0bc416 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.179887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4adbac-7703-4538-917b-50359a0bc416 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.181958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:24.193287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4adbac-7703-4538-917b-50359a0bc416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fd4adbac-7703-4538-917b-50359a0bc416 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:24.198207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:24.200195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:24.201015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:24.204796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 190/383] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:24.206173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:24.232947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:24.301392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b2d5-f8ae-4c40-9d5e-2c7d7bfdeae4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.301687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b2d5-f8ae-4c40-9d5e-2c7d7bfdeae4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.301850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b2d5-f8ae-4c40-9d5e-2c7d7bfdeae4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.303866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 191/384] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:14:24.309453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fbc4e-2e05-495f-bfca-181fbde0c68b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.309760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fbc4e-2e05-495f-bfca-181fbde0c68b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.310000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fbc4e-2e05-495f-bfca-181fbde0c68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.326284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97087cea-0c91-4673-9e86-0d3cb50db93c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:24.331106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-136fbc4e-2e05-495f-bfca-181fbde0c68b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:24.332349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-136fbc4e-2e05-495f-bfca-181fbde0c68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:24.365667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:24.366735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.367213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.367773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.368914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:24.369411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:24.371951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:24.380968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 194/385] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:24.400327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:24.420317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 192/386] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:24.476764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:24.514061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d18c7c-00d6-4e28-b9e7-4bb006265cf5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:24.553912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44058d-c8eb-49ee-a7a1-cc42b03a089a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.554214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44058d-c8eb-49ee-a7a1-cc42b03a089a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.554434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44058d-c8eb-49ee-a7a1-cc42b03a089a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.555771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 193/387] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:14:24.557517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:24.558737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.559023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.559396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.561482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.561823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.562094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.563149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:24.564606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:24.566502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 195/388] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:24 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:24.584410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:24.586868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:24.588273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:24.666573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 194/389] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:24.760706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12729b6-6614-47ac-91a2-f2e154c8c1c8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:24.805812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:24.806482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.806764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.807037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.811242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:24.812778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:24.814821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 196/390] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:24 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:24.820891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:24.919087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:24.927134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:24.932037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:24.951432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0d14-175b-4b6d-a6e0-91a47ddae170 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:24.992508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:24.993183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:24.993460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:24.993867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:24.996207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] RBAC: Authorizing `identity:create_grant(project_id=591bfc2823974e3f9466598de018ddce, group_id=7de07306d2b24d3fa1c654d6cdff0744, role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:24.997664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:25.078595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 195/391] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Sep 3 01:14:24 2022] PUT /identity/v3/projects/591bfc2823974e3f9466598de018ddce/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/3a297fae4df9419b9c53354550781a57 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:25.079535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.094223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacc4ac-b720-4cd0-87e7-d40d96fd814f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.137470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.138546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.139050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.140053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.154978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:25.162283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:25.163268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:25.164082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:25.166927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 197/392] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:25.173036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.284221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d689121-b190-407d-96c2-3c1572905d7d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.291952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:25.305661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:25.332003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df240a1f-c940-4885-bd56-8850434623de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.332919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df240a1f-c940-4885-bd56-8850434623de None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.333377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df240a1f-c940-4885-bd56-8850434623de None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.333686 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1f534-9e10-4d25-a93a-6bc450165aa4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.333983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df240a1f-c940-4885-bd56-8850434623de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.347551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df240a1f-c940-4885-bd56-8850434623de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-df240a1f-c940-4885-bd56-8850434623de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:25.353945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:25.355516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:25.356312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:25.362829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 196/393] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:25.365811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df240a1f-c940-4885-bd56-8850434623de None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.380985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.381992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.381992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.382277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.382948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:25.383247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:25.384675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:25.394904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 198/394] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:25.403775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.545830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df240a1f-c940-4885-bd56-8850434623de None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.616370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a8014-0b1a-4cba-9456-95823b33b1a1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.622383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.623009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.623285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.623555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.624223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:25.624522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:25.625911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:25.650509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 197/395] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:25.655820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.673503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:25.690165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.690885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.691785 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.697373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.703471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:25.704946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:25.708949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 199/396] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:25 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:25.716816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.767494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:25.870604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d3704-9716-4fe5-b655-a9463ed5053b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.907254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647c66-3959-4077-a75b-de6ee8c2ed6a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:25.907590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf5975-1435-4acf-a3a8-6c96e12b37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.908145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf5975-1435-4acf-a3a8-6c96e12b37f4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.908427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf5975-1435-4acf-a3a8-6c96e12b37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.908813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf5975-1435-4acf-a3a8-6c96e12b37f4 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.910023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf5975-1435-4acf-a3a8-6c96e12b37f4 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:25.911512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf5975-1435-4acf-a3a8-6c96e12b37f4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:25.913146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 200/397] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:25 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:25.914331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2fcff-960c-450b-94a5-8073e9373661 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.914735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2fcff-960c-450b-94a5-8073e9373661 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.915007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2fcff-960c-450b-94a5-8073e9373661 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.916584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 201/398] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:25.920762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2fcff-960c-450b-94a5-8073e9373661 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:25.959573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758813e-8bdc-4e68-8981-03ba361f23b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:25.960252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758813e-8bdc-4e68-8981-03ba361f23b1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.961126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758813e-8bdc-4e68-8981-03ba361f23b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.961454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758813e-8bdc-4e68-8981-03ba361f23b1 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:25.962889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758813e-8bdc-4e68-8981-03ba361f23b1 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:25.966018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758813e-8bdc-4e68-8981-03ba361f23b1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:25.968385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 198/399] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:25 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:25.969691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:25.970467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:25.970467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.002125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:26.003724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:26.054788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:26.095558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 199/400] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:26.096865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:26.101751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2fcff-960c-450b-94a5-8073e9373661 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:26.138262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:26.203067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bcc74-dd69-46db-9e9e-2d7d6b876701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:26.203753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bcc74-dd69-46db-9e9e-2d7d6b876701 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.204303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bcc74-dd69-46db-9e9e-2d7d6b876701 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.204303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bcc74-dd69-46db-9e9e-2d7d6b876701 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.214010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8fc7f-881f-49a5-b12c-56cd64b9c50f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:26.215074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bcc74-dd69-46db-9e9e-2d7d6b876701 None admin] RBAC: Authorizing `identity:create_grant(project_id=314c6a4b7a99410c99ac353aff468889, group_id=7de07306d2b24d3fa1c654d6cdff0744, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:26.216548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bcc74-dd69-46db-9e9e-2d7d6b876701 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:26.253526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93070b02-80f2-43f2-803c-0ab4916baf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:26.254168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93070b02-80f2-43f2-803c-0ab4916baf57 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.254478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93070b02-80f2-43f2-803c-0ab4916baf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.254752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93070b02-80f2-43f2-803c-0ab4916baf57 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.255871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93070b02-80f2-43f2-803c-0ab4916baf57 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:26.257332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93070b02-80f2-43f2-803c-0ab4916baf57 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:26.262562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 200/401] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:26 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:26.262562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c3548-f1d8-4b22-ac48-6fc9fa15e5c8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.262562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c3548-f1d8-4b22-ac48-6fc9fa15e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.262562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c3548-f1d8-4b22-ac48-6fc9fa15e5c8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.266652 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 201/402] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:26.267428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c3548-f1d8-4b22-ac48-6fc9fa15e5c8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:26.294119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 202/403] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Sep 3 01:14:25 2022] PUT /identity/v3/projects/314c6a4b7a99410c99ac353aff468889/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:26.295745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.296253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.296718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.317660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:26.319432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:26.398830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 203/404] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:26.400040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:26.432665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c3548-f1d8-4b22-ac48-6fc9fa15e5c8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:26.435254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:26.496339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:26.503024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:26.503792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.504145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.504750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.512710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] RBAC: Authorizing `identity:create_grant(project_id=591bfc2823974e3f9466598de018ddce, group_id=7de07306d2b24d3fa1c654d6cdff0744, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:26.512710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:26.553032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45fa97-0e48-4c8e-b1dd-7b43c009c504 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:26.597756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:26.598565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.598894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.599166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.617479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 202/405] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Sep 3 01:14:26 2022] PUT /identity/v3/projects/591bfc2823974e3f9466598de018ddce/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:26.617641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:26.623138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:26.628374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:26.629183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 204/406] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:26.630719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:26.702482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:26.788743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b0ac4-2281-4829-a276-c6fccb336aea None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:26.861473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:26.864412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.864730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.865107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.866188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:26.866643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:26.869104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:26.882844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 203/407] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:26.888600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:26.894090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48922d-ee18-4dea-84b0-2decaa265086 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:26.918364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:26.929031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:26.956162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:26.956725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:26.957008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:26.957270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:26.988105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:26.993908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:26.994949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:26.995845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:26.998542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 205/408] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:27.005312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:27.044836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff9266-bde1-4183-9f42-b683d06b60ee None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.118455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd529-0f81-4334-9654-f68a6524d399 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.151768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.152556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.152682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.152962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.154086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.155802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:27.156897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.157392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 204/409] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:26 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:27.157957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.162637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.163184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.164094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:27.164618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.166202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:27.168259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:27.177244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 206/410] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:27.188826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:27.274614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:27.291211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:27.301934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677da0f-88fe-4bd6-bb84-50ceb4f61f64 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.342568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.343939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.344962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.345757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.347392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.349478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:27.352288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 205/411] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:27 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:27.358401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:27.421535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec7ec0-c3a1-4cb7-b620-ac0724dbeb3f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.486667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2675cf-1a46-42c3-b95e-9d9c31d8845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.487321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2675cf-1a46-42c3-b95e-9d9c31d8845c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.487588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2675cf-1a46-42c3-b95e-9d9c31d8845c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.487870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2675cf-1a46-42c3-b95e-9d9c31d8845c None admin] PATH_INFO: `/v3/groups/cb778f9298a648b3b9b0d2a605192e9b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.488990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2675cf-1a46-42c3-b95e-9d9c31d8845c None admin] RBAC: Authorizing `identity:get_group(group_id=cb778f9298a648b3b9b0d2a605192e9b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.490493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2675cf-1a46-42c3-b95e-9d9c31d8845c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:27.497378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 207/412] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:27 2022] GET /identity/v3/groups/cb778f9298a648b3b9b0d2a605192e9b => generated 229 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:27.498685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10324-9230-4135-9991-5a84d56d4f08 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.498989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10324-9230-4135-9991-5a84d56d4f08 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.499252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc10324-9230-4135-9991-5a84d56d4f08 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.500755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 208/413] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:27.501467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc10324-9230-4135-9991-5a84d56d4f08 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:27.524141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac366d4-5e47-42cf-9aab-b8c7d483787c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.571200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.572810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.573305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.574727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.577673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] RBAC: Authorizing `identity:create_grant(project_id=314c6a4b7a99410c99ac353aff468889, group_id=7de07306d2b24d3fa1c654d6cdff0744, role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.580109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:27.614256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc10324-9230-4135-9991-5a84d56d4f08 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.615952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:27.649497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f19e-8923-466a-af42-1f7b8f75d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.650133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f19e-8923-466a-af42-1f7b8f75d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.650440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f19e-8923-466a-af42-1f7b8f75d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.650762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f19e-8923-466a-af42-1f7b8f75d1e3 None admin] PATH_INFO: `/v3/projects/06b321cae6a24e3a9217594a16dab365` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.651909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f19e-8923-466a-af42-1f7b8f75d1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=06b321cae6a24e3a9217594a16dab365)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.653353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f19e-8923-466a-af42-1f7b8f75d1e3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:27.659813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 209/414] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:27 2022] GET /identity/v3/projects/06b321cae6a24e3a9217594a16dab365 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:27.659813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.659813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.659813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.679137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:27.686998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:27.687262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 206/415] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Sep 3 01:14:27 2022] PUT /identity/v3/projects/314c6a4b7a99410c99ac353aff468889/groups/7de07306d2b24d3fa1c654d6cdff0744/roles/3a297fae4df9419b9c53354550781a57 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:14:27.692741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:27.742750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:27.848556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 210/416] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:27.875502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b07dd-20c0-4dbf-8f26-d3bdb3dc4251 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:27.911710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:27.912379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:27.912656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:27.912919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] PATH_INFO: `/v3/projects/06b321cae6a24e3a9217594a16dab365/groups/cb778f9298a648b3b9b0d2a605192e9b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:27.914896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] RBAC: Authorizing `identity:create_grant(project_id=06b321cae6a24e3a9217594a16dab365, group_id=cb778f9298a648b3b9b0d2a605192e9b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:27.916337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:28.000545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 207/417] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Sep 3 01:14:27 2022] PUT /identity/v3/projects/06b321cae6a24e3a9217594a16dab365/groups/cb778f9298a648b3b9b0d2a605192e9b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:28.206126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:28.319949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:28.323119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5fadd-602f-4b46-9815-cd2fb9da16ad None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:28.365103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:28.366138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:28.366658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:28.367130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:28.378810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:28.384250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:28.385180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:28.386695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:28.389367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 211/418] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:28.394785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:28.548362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:28.571108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061160e8-244e-433c-87f8-faa1f245862b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:28.610107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:28.610794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:28.611082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:28.611355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:28.612032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:28.612334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:28.613731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4239843-d1fd-4627-b6ce-dcc8bb8d448d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:28.623677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 208/419] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:28.628966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:28.752723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8772d-33f8-4770-9092-94c01f11ebc9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:28.753110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8772d-33f8-4770-9092-94c01f11ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:28.753285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8772d-33f8-4770-9092-94c01f11ebc9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:28.754912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 209/420] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:28.758885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a0b5b-01e6-4fd1-991c-2196a3d3ee27 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:28.763960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:28.764244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:28.764515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:28.795184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aac64f-621d-4dd8-abd2-18dc60e66dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:28.795799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aac64f-621d-4dd8-abd2-18dc60e66dfe None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:28.796086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aac64f-621d-4dd8-abd2-18dc60e66dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:28.796357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aac64f-621d-4dd8-abd2-18dc60e66dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:28.811359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:28.812568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:28.843313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:28.880188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aac64f-621d-4dd8-abd2-18dc60e66dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:28.881687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aac64f-621d-4dd8-abd2-18dc60e66dfe None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:28.949186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 210/421] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:28.991576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 212/422] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:28.998066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.001944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebe430-b1f3-4700-9e30-44a130943c72 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.002227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebe430-b1f3-4700-9e30-44a130943c72 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.002530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebe430-b1f3-4700-9e30-44a130943c72 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.004160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 213/423] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:29.023921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.024208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.024477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.068315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:29.069435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:29.127198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:29.185343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546792b-2a6f-4212-bacb-bca640a2ec16 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.190673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:29.216407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 214/424] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:29.232246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.232889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.233173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.233449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.234604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:29.236098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:29.238085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 211/425] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:28 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:29.243428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.264414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.348994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9955af3-4a07-4772-a9fc-744a060f85e6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.387610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.388645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.389143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.389629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.391206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:29.392860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:29.394967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 215/426] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:29.401194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.467255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:29.470646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b417d-e526-45b8-9c0c-be23a69110da None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.529075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1cbe7-aa44-4c58-884d-efd53017e195 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.543952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.544903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.545456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.545727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.555422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:29.559615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:29.566465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:29.567654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:29.568656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:29.573058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 212/427] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:29.574444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.578774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.579607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.580103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.580538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.581609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=591bfc2823974e3f9466598de018ddce) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:29.582062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:29.584525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:29.592690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 216/428] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=3a297fae4df9419b9c53354550781a57 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:29.598427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.689202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b1fcd-39e3-432f-b23e-0a545d0984af None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.737379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.738095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.738442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.738721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.765642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:29.778860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:29.779954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:29.780750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 213/429] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:29.790249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.848731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106df04-5e72-4ca7-93f6-4a0842c275e6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.880028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:29.892899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.893611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:29.893719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.894004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.894463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.895351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:29.895660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:29.897146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:29.907211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 217/430] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:29.912725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:29.931396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64814af4-9436-445c-9e05-70a11aeea6a0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:29.974970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:29.975507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:29.975789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:29.976056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:29.976731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:29.977029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:29.979408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:29.988971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 214/431] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:29.994738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.035792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e2e08-6cb5-469b-87ef-be894525fef9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.072930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1858f3-6348-4fea-a5de-6a02fbbeba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.074090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1858f3-6348-4fea-a5de-6a02fbbeba6c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.074387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1858f3-6348-4fea-a5de-6a02fbbeba6c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.074811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1858f3-6348-4fea-a5de-6a02fbbeba6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.105841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4e04a-b7c8-42be-afed-eee2321fdf8e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.127972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.131016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.143232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b5d58-7835-48e6-810b-fcdbf1a79570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.143853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b5d58-7835-48e6-810b-fcdbf1a79570 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.144139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b5d58-7835-48e6-810b-fcdbf1a79570 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.144416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b5d58-7835-48e6-810b-fcdbf1a79570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.144807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1858f3-6348-4fea-a5de-6a02fbbeba6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.146598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1858f3-6348-4fea-a5de-6a02fbbeba6c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.214377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b5d58-7835-48e6-810b-fcdbf1a79570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.215847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b5d58-7835-48e6-810b-fcdbf1a79570 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.256844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 218/432] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:14:30.256844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3589df-a52b-41b8-8bc3-c102fa4761cd None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.256844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3589df-a52b-41b8-8bc3-c102fa4761cd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.256844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3589df-a52b-41b8-8bc3-c102fa4761cd None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.259459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 219/433] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:30.260254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3589df-a52b-41b8-8bc3-c102fa4761cd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.326545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 215/434] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:30.328157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.328644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.329112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.349010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:30.350284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:30.352349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.369765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.373515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3589df-a52b-41b8-8bc3-c102fa4761cd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.409568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.410196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.410506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.410781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.411882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.413327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.415147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 220/435] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:30.415813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.430396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 216/436] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:30.430396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.535304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bd94d-d464-45da-82fe-5c4f53fef283 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.535898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07278f81-8026-4d32-b7a6-77bfdbfa9184 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.570002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53151a9-d602-470d-852e-186402c864f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.570661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53151a9-d602-470d-852e-186402c864f6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.570938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53151a9-d602-470d-852e-186402c864f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.571207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53151a9-d602-470d-852e-186402c864f6 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.572037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.572298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53151a9-d602-470d-852e-186402c864f6 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.572658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.572934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.573202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] PATH_INFO: `/v3/projects/591bfc2823974e3f9466598de018ddce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.573952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53151a9-d602-470d-852e-186402c864f6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.574590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] RBAC: Authorizing `identity:get_project(project_id=591bfc2823974e3f9466598de018ddce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.580482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 217/437] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:30.580613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.580688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a982612-ac0a-4496-8f09-e768f1abb7b8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.580688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a982612-ac0a-4496-8f09-e768f1abb7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.580688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a982612-ac0a-4496-8f09-e768f1abb7b8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.580968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 221/438] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/projects/591bfc2823974e3f9466598de018ddce => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:30.581039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 218/439] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:30.582200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.582890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a982612-ac0a-4496-8f09-e768f1abb7b8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.605338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.607005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.688886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0cd3c-d277-4ea9-b7c6-f22cae964603 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.690086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a982612-ac0a-4496-8f09-e768f1abb7b8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.723594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.724128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009ca2f-98d8-4c4a-9118-4db83bdbc761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.724254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.724460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.724729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.724819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009ca2f-98d8-4c4a-9118-4db83bdbc761 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.725025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009ca2f-98d8-4c4a-9118-4db83bdbc761 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.725298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009ca2f-98d8-4c4a-9118-4db83bdbc761 None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.725451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=314c6a4b7a99410c99ac353aff468889) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:30.725757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.726480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009ca2f-98d8-4c4a-9118-4db83bdbc761 None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:30.727218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.727916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009ca2f-98d8-4c4a-9118-4db83bdbc761 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:30.734729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 219/440] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:30.734729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.734729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.734729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.735071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 222/441] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:30.735071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.756184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:30.757731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:30.834948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.846428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 220/442] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:30.846428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.868842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62deb10c-bbf6-4760-b8ca-724cac32551a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:30.871259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:30.915481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:30.916124 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:30.916406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:30.916680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:30.929619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:30.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:30.937066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:30.937877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 223/443] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:30.944741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:30.977290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad881c-49d9-4eaa-802e-f4270e15c5c5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.013200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.013981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.014090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.014356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.015111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=5d8beb80ba7c4b5a9147c7b3d12a6418, scope.project.id=591bfc2823974e3f9466598de018ddce) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:31.015418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.016862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.022853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 221/444] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=591bfc2823974e3f9466598de018ddce&role.id=5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:31.062175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52f5f7-6c99-469b-864b-dd98bc623c14 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.100705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.101335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.101620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.101896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.102609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:31.102918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.104319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.118311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 224/445] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:31.119555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.134713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.144799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.220903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.245433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669197-9157-4f67-b066-1b0518c1cea8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.246314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c375ad-2528-4a34-a940-a547f6704962 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.287193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.287392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.287794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.287794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.287955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.288023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.288203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.288444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.300787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:31.306724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:31.307681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:31.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 225/446] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:31.314365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.376815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.378168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.390032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.427895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2b9dc-c733-4ca7-9bad-c5e32bb38c87 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.467024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.467644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.468057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.468336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.469107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:31.469414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.471101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.478113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.481781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 226/447] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:31.486980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.494555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 222/448] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:31.500382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.607196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36102c9c-9eb6-4406-b2ac-46bed3b4d457 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.620514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eb571-ce6f-44cd-944a-da08fd7eb0ee None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.638191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.648068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.649457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.649740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.650014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.664645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.664645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.664645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.664645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] PATH_INFO: `/v3/projects/314c6a4b7a99410c99ac353aff468889` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.665739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] RBAC: Authorizing `identity:get_project(project_id=314c6a4b7a99410c99ac353aff468889)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.667343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.669537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 223/449] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/projects/314c6a4b7a99410c99ac353aff468889 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:31.674964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.731251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.731456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.732718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.785189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7174b-804f-4ed1-9a95-74c80c8416f3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.819162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.819823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.820105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.820372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] PATH_INFO: `/v3/groups/7de07306d2b24d3fa1c654d6cdff0744` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.821535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] RBAC: Authorizing `identity:get_group(group_id=7de07306d2b24d3fa1c654d6cdff0744)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.823022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.825431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/groups/7de07306d2b24d3fa1c654d6cdff0744 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:31.831218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.841732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 227/451] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:14:31.847781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:31.878976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.933777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b9146-fa58-419d-80fe-ef5a54e18e2b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.951244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:31.951869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef14e46-ac43-4e6f-896e-3be86679acaa None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:31.968287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.968894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.969174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.969444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.970183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] RBAC: Adding query filter params (group.id=7de07306d2b24d3fa1c654d6cdff0744, role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=314c6a4b7a99410c99ac353aff468889) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:31.970798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.972586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.978670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 225/452] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/role_assignments?group.id=7de07306d2b24d3fa1c654d6cdff0744&scope.project.id=314c6a4b7a99410c99ac353aff468889&role.id=3a297fae4df9419b9c53354550781a57 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:31.989883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:31.990570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:31.990852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:31.991121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] PATH_INFO: `/v3/projects/06b321cae6a24e3a9217594a16dab365` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:31.992320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] RBAC: Authorizing `identity:get_project(project_id=06b321cae6a24e3a9217594a16dab365)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:31.993813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:31.996162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 228/453] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:14:31 2022] GET /identity/v3/projects/06b321cae6a24e3a9217594a16dab365 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:32.002007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:32.107332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb88af6-4e17-49fe-835e-f717b39a51c2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:32.145529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4feefb-b815-43ac-8484-7167f99a078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:32.146123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4feefb-b815-43ac-8484-7167f99a078f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:32.146462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4feefb-b815-43ac-8484-7167f99a078f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:32.146730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4feefb-b815-43ac-8484-7167f99a078f None admin] PATH_INFO: `/v3/groups/cb778f9298a648b3b9b0d2a605192e9b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:32.147839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4feefb-b815-43ac-8484-7167f99a078f None admin] RBAC: Authorizing `identity:get_group(group_id=cb778f9298a648b3b9b0d2a605192e9b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:32.149293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4feefb-b815-43ac-8484-7167f99a078f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:32.150891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 226/454] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Sep 3 01:14:32 2022] GET /identity/v3/groups/cb778f9298a648b3b9b0d2a605192e9b => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:32.157866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:32.259206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:32.265700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74825c4-0bde-4bb1-a5ea-746c3d89c35d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:32.300777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:32.301426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:32.301691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:32.302233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:32.303146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] RBAC: Adding query filter params (group.id=cb778f9298a648b3b9b0d2a605192e9b, role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=06b321cae6a24e3a9217594a16dab365) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:32.303781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:32.305228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feaa98-0d7b-4c04-a8e1-fbcb15b791b9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:32.311236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 229/455] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Sep 3 01:14:32 2022] GET /identity/v3/role_assignments?group.id=cb778f9298a648b3b9b0d2a605192e9b&scope.project.id=06b321cae6a24e3a9217594a16dab365&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:34.800961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb2d9b-9286-4bcc-afb4-b4e4a6a862d0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:34.801966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb2d9b-9286-4bcc-afb4-b4e4a6a862d0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:34.802843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb2d9b-9286-4bcc-afb4-b4e4a6a862d0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:34.804294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 227/456] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:34.810822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148546c1-9853-4075-ad5c-39b26b3a1510 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:34.811252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148546c1-9853-4075-ad5c-39b26b3a1510 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:34.811438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148546c1-9853-4075-ad5c-39b26b3a1510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:34.821200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1244f-7687-4c5f-8e2f-bf2ceaf27079 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:34.821761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1244f-7687-4c5f-8e2f-bf2ceaf27079 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:34.822251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1244f-7687-4c5f-8e2f-bf2ceaf27079 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:34.824217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 228/457] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:34.829517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:34.830047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:34.830557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:34.849081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-148546c1-9853-4075-ad5c-39b26b3a1510 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:34.850720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-148546c1-9853-4075-ad5c-39b26b3a1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:34.864965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:34.866485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:34.959191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 230/458] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:34.972624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8621e-9a1c-4db6-a767-a371d8708683 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:34.972903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8621e-9a1c-4db6-a767-a371d8708683 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:34.973169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8621e-9a1c-4db6-a767-a371d8708683 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:34.974520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 231/459] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:14:34.978625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:34.979778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32db9b-f1a7-40fe-a76c-7b85be2cca9f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:34.979980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32db9b-f1a7-40fe-a76c-7b85be2cca9f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:34.980279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32db9b-f1a7-40fe-a76c-7b85be2cca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.011425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 229/460] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:35.012169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ed32db9b-f1a7-40fe-a76c-7b85be2cca9f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:35.013519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed32db9b-f1a7-40fe-a76c-7b85be2cca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:35.093014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 232/461] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:35.310057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.358079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda45d5-8765-4098-85ca-77fd3bf37845 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.358392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda45d5-8765-4098-85ca-77fd3bf37845 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.358602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda45d5-8765-4098-85ca-77fd3bf37845 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.360223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 233/462] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:35.375717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda45d5-8765-4098-85ca-77fd3bf37845 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.405569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:35.430150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623adc8-36ab-498f-91a9-12385056ac64 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:35.466661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:35.467568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.467847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.468104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.478987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:35.481464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:35.491608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda45d5-8765-4098-85ca-77fd3bf37845 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-08d29009-aa47-459d-b30d-fc6fc644060c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:35.498540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:35.499425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:35.500272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:35.500272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:35.500272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:35.500272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 230/463] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:35.501263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.501377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.501724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.519854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:35.521307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:35.530262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:35.530924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.531204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.531473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.541629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:35.543902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:35.548645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:35.558495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:35.563588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:35.564726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:35.564726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:35.564726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:35.564726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 234/464] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:35.564726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.594794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 231/465] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:35.595940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.644603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:35.678410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4470d45-1b33-4260-8281-cbc4b2770e49 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:35.718699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:35.719707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.720167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.720618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.733390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:35.735369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:35.738736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be38699-ecf2-424d-930f-16533671bc7d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:35.752948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:35.753969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:35.755195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:35.755195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:35.755195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:35.761967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 235/466] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:35.763100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.785541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:35.787300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.787589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.787872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.788940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:35.789259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:35.790754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:35.814146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 232/467] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:35.815112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.852318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:35.896306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3fbe9-032e-4675-9db7-e88574c561f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:35.921498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:35.933546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:35.936615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.936615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.936816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e6e2f-47a5-493e-96d4-1d45a001b57b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:35.936914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.936914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:35.936914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:35.937151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:35.956640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 236/468] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:35.956640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:35.976034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:35.976645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:35.976931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:35.977198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:35.977838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:35.978139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:35.980216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:35.991986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 233/469] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:35.993012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:36.069091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849a520-6269-4a15-bb88-5e35ce9a98b1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:36.076500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:36.094474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6bc83-5f13-441e-9ce8-04d3bfd9b03a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:36.107249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:36.107860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.108397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.108397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.109040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:36.110533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:36.132582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:36.136749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.136749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.136749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.144618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:36.146403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:36.161529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:36.162526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:36.163380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:36.163380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:36.163380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:36.163380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 234/470] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:36.163380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:36.200054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 237/471] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:35 2022] POST /identity/v3/users => generated 246 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:36.201313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:36.262715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:36.296133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486aedfa-bbf8-46ee-b367-103f02723031 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:36.331950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c5ec8-876d-418d-aeba-7f1edd3ed4c8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:36.333834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:36.334497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.334776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.335047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.335708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:36.337201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:36.346513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:36.370850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee6c97-3a9e-4882-b7cc-519afb2bbd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:36.371469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6c97-3a9e-4882-b7cc-519afb2bbd57 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.371754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6c97-3a9e-4882-b7cc-519afb2bbd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.372027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee6c97-3a9e-4882-b7cc-519afb2bbd57 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.372675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee6c97-3a9e-4882-b7cc-519afb2bbd57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:36.374120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee6c97-3a9e-4882-b7cc-519afb2bbd57 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:36.440116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 235/472] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:36 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:36.440883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:36.494625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 238/473] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:36 2022] POST /identity/v3/users => generated 248 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:36.498273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594edd2-070c-4700-95cf-67b9602ff09b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.498852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594edd2-070c-4700-95cf-67b9602ff09b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.499499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594edd2-070c-4700-95cf-67b9602ff09b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.501337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 239/474] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:36.508706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.509263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.509781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.551053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:36.553318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:36.633707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb5cf1-5715-48b3-9e01-20418c2f355b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:36.650913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 240/475] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:36.679225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:36.680115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.680538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.680952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.681796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:36.682237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:36.684161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:36.700528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 236/476] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:36.706000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:36.737744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:36.817442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f0271-dfb8-43f3-8da1-64de4eb31bc0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:36.855603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:36.856498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:36.856920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:36.857332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:36.858118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:36.859752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:36.938112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 241/477] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:36 2022] POST /identity/v3/users => generated 249 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:36.968756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:37.027730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:37.092133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca9284-8075-4051-8bc3-a6af865f2aad None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:37.144524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:37.145417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:37.145700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:37.145969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:37.146691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:37.148110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:37.199643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.sql.core [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 01:14:37.199643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 03 01:14:37.199643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [parameters: {'id': '280a8449e59f43f89d7034905e7a98fc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 03 01:14:37.199643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81627) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Sep 03 01:14:37.216303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return ref.to_dict() Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 01:14:37.217209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application yield resource Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise self.value Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application session.commit() Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 01:14:37.218973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.flush() Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application compat.raise_( Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 01:14:37.219861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 01:14:37.220696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 01:14:37.221526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [parameters: {'id': '280a8449e59f43f89d7034905e7a98fc', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:37.223259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 03 01:14:37.225430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:37.227537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 237/478] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Sep 3 01:14:36 2022] POST /identity/v3/roles => generated 144 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:37.227674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:37.333474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:37.353517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9e86f-4941-4548-a185-79205d98b559 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:37.393772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427d995-0e11-492d-ae70-fc4f81f1bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:37.394464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427d995-0e11-492d-ae70-fc4f81f1bb07 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:37.394755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427d995-0e11-492d-ae70-fc4f81f1bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:37.395027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427d995-0e11-492d-ae70-fc4f81f1bb07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:37.407504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427d995-0e11-492d-ae70-fc4f81f1bb07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0427d995-0e11-492d-ae70-fc4f81f1bb07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:37.412417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:37.413272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:37.414761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 242/479] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:37.419737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:37.560841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800006-e694-4072-bf1a-5405b42f2e3a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:37.575454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:37.601852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:37.602497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:37.602777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:37.603046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:37.603712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:37.604108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:37.606047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dcbba-5a2d-4042-a1ca-169dbadc50fb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:37.618419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 238/480] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:38.395945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4846946-029c-4658-9089-f60ee2f21f6d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:38.396880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4846946-029c-4658-9089-f60ee2f21f6d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:38.397336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4846946-029c-4658-9089-f60ee2f21f6d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:38.399122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 243/481] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:38.405567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:38.406044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:38.406134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:38.435294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:38.436491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:38.534453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 239/482] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:38.852442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d86ab4-68f6-498f-963e-6be1208ae036 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:38.852761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d86ab4-68f6-498f-963e-6be1208ae036 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:38.852993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d86ab4-68f6-498f-963e-6be1208ae036 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:38.855321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 244/483] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:38.858928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:38.877240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed6a3-56d7-4097-9643-f9f32b567bb1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:38.877540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed6a3-56d7-4097-9643-f9f32b567bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:38.877789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed6a3-56d7-4097-9643-f9f32b567bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:38.903461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d91ed6a3-56d7-4097-9643-f9f32b567bb1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:38.904892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d91ed6a3-56d7-4097-9643-f9f32b567bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:38.981253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0ac3e-3b76-4d60-b931-d31846ef02e6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:38.998090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 245/484] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:38.999381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48701b2-5483-4a6d-a571-290e389df8e5 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:38.999664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48701b2-5483-4a6d-a571-290e389df8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:38.999937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48701b2-5483-4a6d-a571-290e389df8e5 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.001428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 246/485] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:39.007559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.007850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.008125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.027197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:39.028393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:39.038688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.103125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.103691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.103969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.104971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.126656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:39.137691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:39.138607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:39.139479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:39.142370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 240/486] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:39.144342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.162979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 247/487] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:39.272723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3791c-6124-4611-9b4f-40c549b50168 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.308289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.308909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.309189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.309459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.310129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:39.310454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:39.311844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:39.322820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 241/488] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:39.325250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.334717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.336022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.403397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.445574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5e4f8-444a-4b2f-9df3-8e310fff0eaf None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.483484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.484274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.484561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.484832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.496546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:39.501722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:39.502637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:39.503905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 248/489] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:39.515278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.555921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228cb5e-dd1b-48c8-a480-f2a967260776 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.589100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.597533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.598549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.599429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.599705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.613995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:39.615900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:39.636085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:39.637155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:39.638741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:39.638741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:39.638741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:39.639074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbc045-6c26-4282-93c5-f5cf6d6b9f48 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.639219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 242/490] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:39.642503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.681684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.682209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.683162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.683392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.684060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:39.684358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:39.686183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:39.698460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 249/491] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:39.699631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.732589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.754554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d91f09-119e-4e3b-9036-b512aa237e76 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.792365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.793120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.793400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.793671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.807085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:39.813586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:39.814588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:39.815376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:39.815979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7dcbd-8078-4c09-8097-cfe3dfd3226a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.816053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 243/492] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:39.816053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.845025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.856987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.857877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.858254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.858558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.859300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:39.859601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:39.860974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e082bcc-808a-4203-98f5-3c399890d276 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:39.874404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 250/493] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:39.875888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0bc4-3fb3-4323-aea1-6d511d3c1501 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.876241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0bc4-3fb3-4323-aea1-6d511d3c1501 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.876521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b0bc4-3fb3-4323-aea1-6d511d3c1501 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.881787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 251/494] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:39.882763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0bc4-3fb3-4323-aea1-6d511d3c1501 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:39.937170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a28b1-1c8f-42c6-8b09-0c6a50aa7079 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:39.975544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bb27f-176b-439a-af34-185fe407735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:39.976217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bb27f-176b-439a-af34-185fe407735b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:39.976457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bb27f-176b-439a-af34-185fe407735b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:39.976729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bb27f-176b-439a-af34-185fe407735b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:39.986762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:39.989482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bb27f-176b-439a-af34-185fe407735b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:39.991081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bb27f-176b-439a-af34-185fe407735b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.001054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b0bc4-3fb3-4323-aea1-6d511d3c1501 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-768bb27f-176b-439a-af34-185fe407735b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:40.006054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:40.006952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:40.008565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:40.008565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:40.008565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:40.008565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 244/495] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:40.012935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bb27f-176b-439a-af34-185fe407735b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.038743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.039350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.039769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.040044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.041019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:40.041633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.043462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2f9f-80ac-471d-892b-270b0ef69c8c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.058046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 252/496] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:40.058046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.058046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.058046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.076162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:40.077369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:40.122307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bb27f-176b-439a-af34-185fe407735b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.122918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.150630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 253/497] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:40.152145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.171277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.172920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.173209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.173480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.186295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.188122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:40.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:40.203449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:14:40.204329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:40.208474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 245/498] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:40.209535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.275492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.279584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909cbb4-2a04-444d-a44a-c11abd47b80a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.318090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.318731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.319014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.319285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.319934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:40.320236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.321608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.325912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf1d2b-7923-4dcd-8e6b-15853811a623 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.335388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 254/499] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:40.336188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.366159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.366946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.367241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.367516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.377985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.378640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.380104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-64f17dfc-e502-4931-a2ed-385590dd8171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:40.394173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:40.395525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:40.396373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:40.396373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:40.396373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:40.398644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 246/500] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:40.400614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1641-b137-4eda-aca3-3735a6799328 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.400906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1641-b137-4eda-aca3-3735a6799328 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.401865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1641-b137-4eda-aca3-3735a6799328 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.403169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 247/501] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:40.406439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1641-b137-4eda-aca3-3735a6799328 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.451595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2ba83-19dc-4e3e-890e-f7429209140b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.485596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.486158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.486486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.486758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.487482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:40.487789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.489223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.504559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1641-b137-4eda-aca3-3735a6799328 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.507012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.527245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 255/502] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:40.528074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.543436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.544361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.544641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.544916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.554303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.556297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0e173506-b5d7-48df-9648-63f803bead59 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:40.570690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:40.571836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:40.572768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 248/503] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:40.573782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.574153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.574472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.593318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:40.594566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:40.625696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.632277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef727c8-405a-4099-875b-37a1a1fe3b9a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.674371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 249/504] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:40.674371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.674652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.674652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.674652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.674652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.674980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:40.674980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.674980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.681553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 256/505] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:40.682997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.777634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.785192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949241e-04eb-4b81-bff0-74b19721c8fe None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.796967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a1fc0-e48b-45d4-9e68-7b8f08431ee1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.828341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.828987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.829274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.829543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.835390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.835999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.836277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.836545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.840518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:40.845299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:40.846168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:40.847034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:40.851742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 250/506] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:40.852454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:40.866093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:40.914317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:40.915867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:40.961821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcaa0cf-94e1-45a8-ba41-ae5836bbcecf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:40.997257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:40.997868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:40.998163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:40.998462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:40.999183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:40.999481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.000918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.016306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 257/507] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:41.017152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.037112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 251/508] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Sep 3 01:14:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:41.037871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.052602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.074771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.118632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2a702-d2e1-4451-9df5-e6e03c54e51c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.136376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d0cfb-61c5-461b-934c-5eb12b024598 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.152637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.153265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.153543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.153809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.161007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.162464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.169764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.170389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.170669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.170937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.171586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:41.171885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.173265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:41.175095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:41.176135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 258/509] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:41.177219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.185060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 252/510] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:41.185873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.263433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.277687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca3981-5fdd-4a28-a0b0-2585ffaa550e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.288832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd115-844f-43fc-a60d-87cb72e48d5c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.291229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.311180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.311811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.312096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.312370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.322256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.323438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.323714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.323870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.323995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.325438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.334119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.335580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:41.340140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:41.341055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:41.341900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:41.341900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.341900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:41.341900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 259/511] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:41.342122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:41.351559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:41.352551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:41.353411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:41.353411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.353411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:41.353411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 253/512] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:41.353411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.445161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee1008-61a3-4800-bde8-5f12a518da47 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.452441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b499dfe-9e0a-4832-a31f-ecd4f66546cb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.477388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.477525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.478103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.478352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.478640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.486893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.487497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.487776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.488043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.488765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:41.489069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.490530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.507984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 254/513] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:41.508788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.513928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.547709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.549133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.606487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27123919-78ac-49ce-80c4-5eee472ee4c7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.640262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.640893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.641176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.641447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.653226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 260/514] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:41.653226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.653432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.654130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:41.669129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:41.670081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:41.670951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:41.670951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.670951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:41.670951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 255/515] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:41.671331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.705866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.718133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.752892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e39d1-8661-4347-8091-c3cc09933d1a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.769803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b10d1d-cdd1-48c1-9be8-2603c3b18aa5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.786787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.787391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.787671 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.787941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.788581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:41.788880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.790242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.806284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 261/516] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:41.807238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.809174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.809797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.810091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.810562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.811188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:41.811500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.812910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.827397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 256/517] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:41.828782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:41.910217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07623a-bb83-4c22-b0b5-f44e9f7a6c60 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.924970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.930476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6dd70-3a06-4134-8566-96e5d6241a3c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:41.936655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:41.944128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.944739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.945019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.945289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.964199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012503fa-a513-4438-bc16-4711779edfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:41.964823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012503fa-a513-4438-bc16-4711779edfae None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:41.965101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012503fa-a513-4438-bc16-4711779edfae None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:41.965371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012503fa-a513-4438-bc16-4711779edfae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:41.975620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012503fa-a513-4438-bc16-4711779edfae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:41.977075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012503fa-a513-4438-bc16-4711779edfae None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-012503fa-a513-4438-bc16-4711779edfae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:41.991502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:41.992586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:41.999029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:41.999029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:41.999029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:41.999029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 257/518] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:41.999029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012503fa-a513-4438-bc16-4711779edfae None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.022417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.023877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.090489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012503fa-a513-4438-bc16-4711779edfae None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.120409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 262/519] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:42.121378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.123475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.124080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.124352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.124694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.125338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:42.125636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.127029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.139329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 258/520] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:42.140133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.150126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.153757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.227170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9a1b0-5274-4be6-97c7-581f26c499e3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.240286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7610f-fb40-4174-bc03-cfcb9e99c159 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.264171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.264779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.265062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.265332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.265986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.267547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.278143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.278813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.279097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.279371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.286030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.290458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:42.299715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:42.300712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 259/521] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:42.306672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.335994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 263/522] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:14:42 2022] POST /identity/v3/users => generated 247 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:14:42.336824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.375076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.413632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2456c-d696-48ea-b95f-fe486fd3dbaf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.435762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.452491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.453110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.453410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.453689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.454373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:42.454667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.456095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.457093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421344e-abbd-423f-8d5d-500d993f264a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.474474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 260/523] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:42.474474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.500703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.501336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.502430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.502909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.514263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.516470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:42.539233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:42.540327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:42.541181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:42.541181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:42.541181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:42.541181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 264/524] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:42.541408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.597839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db8690-ea31-49b2-af85-f68c74708d58 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.634237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.642371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.643005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.643284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.643582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.649352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f040300-7f33-4317-8c8c-215b05ab9346 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.651426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.653011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:42.664905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:42.665844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 01:14:42.667399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:42.670788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 261/525] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:42.671848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.689781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.690443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.690727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.691002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.691706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:42.692016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.693563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.695405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.712673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 265/526] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:42.714303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.781850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111a7cc-1956-434b-b46f-79e7c15d5e33 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.822381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.822381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.822674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.823531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.826147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4d6fd-9482-4d3f-a401-9d79d1d9660a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.831143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.832605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:42.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:42.846471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 262/527] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:42.851965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.868246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:42.868859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:42.869147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:42.869406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:42.870320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:42.870652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:42.872070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:42.884505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 266/528] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:42.885784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:42.910872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.947818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:42.968107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6715fd-793a-43c4-8ce9-0787bffe4c7f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:42.994301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7651e84-e9da-461f-a12c-79635d04186c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.005779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.006719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.007005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.007430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.008162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:43.008466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.009959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.028718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 263/529] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:43.029661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.035480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.036315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.036594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.036859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.037585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=1749220d9e9547eba7ea035508c365bd) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:43.037889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.039350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.045806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 267/530] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:42 2022] GET /identity/v3/role_assignments?user.id=1749220d9e9547eba7ea035508c365bd&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:43.055424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.171297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c38cd4-5229-4ab8-a175-0e7478d0d3ff None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.180127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:43.207528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc3ca3-5ccb-490a-8d24-6bf5d7c34412 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.216688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.217304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.217585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.218280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.218588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:43.218890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.220358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.231047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:43.244267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 264/531] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:43.245384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.256817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.258057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.258363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.259181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.268196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.270945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:43.287477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:43.288359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:43.289194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 268/532] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:43.290278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.360651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563675b-be92-4d80-91e4-38e301a0db56 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.398900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.399582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.399866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.400138 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.428342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4544dc-671a-4e1f-9ccb-5fb3798f719b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.437740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:43.474468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.474681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.475420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.475843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.476154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.476433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.477302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=cbc9f7e49e5946f48df125d4efd80dfc) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:43.477758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.481814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.489215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 269/533] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/role_assignments?user.id=cbc9f7e49e5946f48df125d4efd80dfc&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:43.490654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.574310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:43.606377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 265/534] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:43.610915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.670059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351820f8-bd32-43c2-910e-28b26182efa8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.725050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.725584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.725869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.726146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.726905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:43.727346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.729862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:43.730285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.750069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 270/535] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:43.751341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.809353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a04989b-200c-4274-8b95-86273d8e8a4a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.855023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:43.855674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:43.855954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:43.856228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:43.874048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:43.876348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:43.909867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:43.910964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:43.911816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:43.911816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:43.911816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:43.912407 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 266/536] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:43.918986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:43.956593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56952d0d-3871-43dd-8b58-346e82cbbc3c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:43.959180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:44.025770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:44.026430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.026704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.027093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.027839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=edb970ecb499474f87e02b9ee49c8c8a) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:44.028151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:44.030457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:44.041264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 271/537] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/role_assignments?user.id=edb970ecb499474f87e02b9ee49c8c8a&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:44.088427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897054c5-62a8-497f-9624-83f952cd4526 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:44.107165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:44.125800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:44.126467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.126954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.127236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.127912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:44.128214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:44.129764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:44.149815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 267/538] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:44.156325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:44.304100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c464617-2dca-4e7a-810c-5b354db92c8e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:44.340577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:44.341314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.341595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.341866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.350380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:44.351928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-246fb691-482c-4d1f-9ec1-38c164b55e6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:44.364212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:44.365069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:44.365929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 272/539] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:44.371697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:44.503964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3b8eb-35f8-44db-ad3c-692abb4968c0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:44.536944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb720f7-6a89-48d3-862c-9b146b0936c2 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.537283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb720f7-6a89-48d3-862c-9b146b0936c2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.537500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb720f7-6a89-48d3-862c-9b146b0936c2 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.539053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 273/540] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:14:44.545327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79acfb-219e-4849-9cde-30eaa943ffb9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.545834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79acfb-219e-4849-9cde-30eaa943ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.546300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79acfb-219e-4849-9cde-30eaa943ffb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.560810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:44.561364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.561649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.561937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.562663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:44.562975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:44.564437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:44.567186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7f79acfb-219e-4849-9cde-30eaa943ffb9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:44.568765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7f79acfb-219e-4849-9cde-30eaa943ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:44.587461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 268/541] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:44.595506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:44.596102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:44.646104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 274/542] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:44.686926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:44.762034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75c592-0590-4cfa-a12f-c6d28f504086 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:44.800770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:44.801391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:44.801670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:44.801945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:44.802773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=d50713b3e4864430a69d0e00068257de) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:44.803176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:44.809941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:44.820316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 269/543] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:44 2022] GET /identity/v3/role_assignments?user.id=d50713b3e4864430a69d0e00068257de&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:45.143244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630c3b-fcea-437f-8edf-01bf448d4b25 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:45.143586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630c3b-fcea-437f-8edf-01bf448d4b25 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:45.143703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca630c3b-fcea-437f-8edf-01bf448d4b25 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:45.145532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 275/544] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:45.146546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:45.150445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:45.150738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:45.151611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:45.171088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:45.172299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:45.276943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34600177-f45b-491b-a3b6-24b3863b057c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:45.278656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:45.317428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:45.318081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:45.318467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:45.318664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:45.331328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:45.340362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:45.341247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:45.342050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 270/545] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:45.349736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:45.372376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 276/546] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:14:45.497419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8611335-1d00-4d75-8ad0-3d4b0bbba478 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:45.544713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:45.545340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:45.545626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:45.545903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:45.546600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:45.546910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:45.548354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:45.561750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 271/547] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:45.567319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:45.649296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:45.678660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:45.710304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:45.737159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1726-eb10-4120-ab27-4cf91fb65f53 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:45.774134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:45.774709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:45.774986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:45.775252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:45.788017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:45.790526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:45.820424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:45.821347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:45.823177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:45.823177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:45.823177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:45.824574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 277/548] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:45.830157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:45.830717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf89df1-52c6-4c7d-aaba-dfef042e79c5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:45.868716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:45.869770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:45.870275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:45.870796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:45.881857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:45.888101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:45.889169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:45.889965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:45.892603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 272/549] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:45.898242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.018758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.036368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3feb6-e62f-4d9a-9a82-1d0d9286e6fc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.051395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ef178-1e0d-40e9-8707-894db92fd8ff None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.086712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.087503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.087790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.088063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.089343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:46.089861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.091301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.100118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.105888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 278/550] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:46.115841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.120546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.121165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.121441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.121705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.122395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:46.122696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.124299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4be163-ef70-4d34-a53a-6137864c8dc4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.133451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 273/551] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:46.134785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2e91-086e-494f-bf2a-2c590d2ad354 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.135069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2e91-086e-494f-bf2a-2c590d2ad354 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.135475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2e91-086e-494f-bf2a-2c590d2ad354 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.137008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 274/552] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:46.141706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2e91-086e-494f-bf2a-2c590d2ad354 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.244404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11eeb-ad07-45fd-9f85-67de5b559fff None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.273503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2e91-086e-494f-bf2a-2c590d2ad354 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.284498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.285413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.285895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.286456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.294263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.295989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9855856d-40e8-4a17-aef4-e101c4ba900e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:46.308798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:46.309747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:14:46.310701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:46.316860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 279/553] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:46.316860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.316860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.316860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.319276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.319914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.320194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.320462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.335088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.336573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.338085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:46.340720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:46.345979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:46.357835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:46.369202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:46.370112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:46.370112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:46.370112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:46.370112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 275/554] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:46.370112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.411396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.434422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 280/555] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:46.435637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.492341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb900a-6a7d-4cfa-af52-4d935e6b90f2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.531504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.532441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.537430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.537430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.537430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:46.537430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.537430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a266ec-9908-4051-9694-9a2fe7b00ace None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.555728 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 276/556] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:46.562017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381ad7b-221c-4e5f-9e73-5f49d43e980c None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.568092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381ad7b-221c-4e5f-9e73-5f49d43e980c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.568092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381ad7b-221c-4e5f-9e73-5f49d43e980c None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.568092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 277/557] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:46.568092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381ad7b-221c-4e5f-9e73-5f49d43e980c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.577194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c6cf6-3d15-4182-a94e-b314bc1900cc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.616661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.617328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.617699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.617894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.618577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:46.618884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.620943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff46288-2db6-4e6d-ac80-98a5a1385cca None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.632148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.635050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 281/558] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:46.637026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.637393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.637584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.657411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:46.658716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:46.683232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381ad7b-221c-4e5f-9e73-5f49d43e980c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.710205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.720554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.723086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.723086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.723086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.735289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 282/559] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:46.736140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.805030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.806588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.852621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9808872e-e5d8-47b8-9134-abff8fad7686 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:46.872989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:46.890895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:46.891837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:46.892357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:46.892842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:46.901398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:46.903132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:46.917958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:46.918967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:14:46.919915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:46.925646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 278/560] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:14:46.925646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.927685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 283/561] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:46.932854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:46.982165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.050699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa4d2e-04fe-40fe-8bd2-da38416f30c8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.070980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bae002-cbdf-44a9-8cd9-d10827f0cd8b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.083962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.084593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.084872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.085141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.096181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.097632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.110822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.111463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.111745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.112023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:47.113549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:47.114697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:47.115627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:47.115627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:47.115627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:47.115627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 284/562] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:47.115627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.125425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:47.130308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:47.131460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:47.132473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 279/563] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:47.134605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.153357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.208601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.218778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eed4-b3d4-4b3f-b42b-b3dbe2e959d5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.233709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4b729-366a-490a-b66f-b79090562fad None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.252568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.253177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.253451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.253719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.254466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:47.254770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.256189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01c677-8bbe-413f-ae89-78a6091f7fa2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.267212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.267855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.268152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.268410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.273945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 285/564] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:47.275096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8ed66-70b3-4ef0-8ef0-eaf71725cf8d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.275376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8ed66-70b3-4ef0-8ef0-eaf71725cf8d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.275726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8ed66-70b3-4ef0-8ef0-eaf71725cf8d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.278389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 286/565] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:47.278389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8ed66-70b3-4ef0-8ef0-eaf71725cf8d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.281800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:47.286730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:47.287846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 280/566] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:47.289106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.372446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.391825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8ed66-70b3-4ef0-8ef0-eaf71725cf8d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.395564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d0123-a944-4224-b862-8a4615924d31 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.426173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.426803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.427078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.427344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.427975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:47.428273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.429629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.429731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.430231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.430540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.430821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.431477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:47.431775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.433262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a6377-5b61-4ccd-9bb5-4a003b0787f5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.440805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 287/567] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:47.445226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 281/568] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:47.445355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.445433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.445433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.445433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.457036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.461243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:47.462457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:47.528622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 282/569] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:47.529656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.548842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98a059-3125-460d-9be1-9670337fc87a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.586644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.587267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.587544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.587838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.599363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.600800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.602176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:47.615327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:47.616292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:47.617185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:47.617185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:47.617185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:47.617185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 288/570] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:47.617185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.643093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d36ff3-f236-486c-bcf7-048e59d42cbd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.679013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.679624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.679903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.680170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.680825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:47.681122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.683073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.686478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.691117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 283/571] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:47.691919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.738917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cb23b-5c8e-41f8-aa21-34aec4f401db None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.778811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.779448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.779728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.779999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.791297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.793359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.796450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06448fe0-951f-42e9-8c24-4adf4ce6f660 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:47.808310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:47.809273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:47.810147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:47.810147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:47.810147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:47.810147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 289/572] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:47.811205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.834559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.840874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.841861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.845464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.845464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.849048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.850627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:47.866591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:47.868131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:47.869221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:47.871176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 284/573] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:47.872039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:47.924088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75c06a-bc6f-45d3-8252-7ad91de042be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.936610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:47.957648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:47.958252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:47.958563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:47.958840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:47.959479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:47.959790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:47.961164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:47.972058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d10a92-a3d2-4ce1-84a7-dcceec7fb279 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:47.974010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 290/574] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:47.978553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.011197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.012054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.012337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.012607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.023590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.025047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:48.039577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:48.040535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:48.041479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:48.041479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:48.041479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:48.045440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 285/575] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:48.046164 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.077402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df1ed3-5bc1-415c-a04d-44948f09ee92 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.087410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.110766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.111476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.111762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.112030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.122421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:48.127122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:48.128058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:48.128889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 291/576] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:48.129565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.148519 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9154d7-c210-4db8-983b-fe611edec8ec None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.160908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.181369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.182496 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.182930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.183205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.183846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:48.184141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.185507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.196940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 286/577] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:48.197794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.230564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b6446-283e-45b3-a8de-03b9e61b7adb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.264100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.264701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.264977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.265244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.265976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:48.266275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.267737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.279249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 292/578] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:48.280031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.299363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5b7d6-be77-4911-89a0-39ea59d895d2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.300882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.334782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.335402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.335680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.336034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.343221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.344657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:48.356339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:48.357408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:48.358471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 287/579] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:48.359749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.373142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.381566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2792340-dcef-4a91-b492-e630fbe953e6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.415657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.416411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.416744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.417133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.417769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:48.418075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.419504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.430967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 293/580] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:48.431812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.459291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3bdf36-d5dd-4356-942d-aa2b2bc323a6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.493594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.494206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.495047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.495325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.495988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:48.496291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.497671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.505426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 288/581] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:48.506216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.529639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de41936-5645-41a6-91c4-4172062d3dce None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.534573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.563463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67666952-7896-4717-b934-482019010632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.564085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67666952-7896-4717-b934-482019010632 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.564362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67666952-7896-4717-b934-482019010632 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.564624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67666952-7896-4717-b934-482019010632 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.571361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67666952-7896-4717-b934-482019010632 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.572874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67666952-7896-4717-b934-482019010632 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-67666952-7896-4717-b934-482019010632 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:48.583992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:48.585030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 294/582] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:48.586031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67666952-7896-4717-b934-482019010632 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.600498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.611011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea5dfc-52d4-4fd5-a6fd-9b3c807ddc48 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.643904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.644505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.644788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.645060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.646051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=0711b98a2c214b7cb807f2d211b628c7) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:48.646379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.647827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.653596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 289/583] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/role_assignments?user.id=0711b98a2c214b7cb807f2d211b628c7&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:48.654422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.696360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67666952-7896-4717-b934-482019010632 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.738426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.739314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.739956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.739956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.740540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:48.740845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.742309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.755098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 295/584] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:48.760668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.777065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.790305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd14382-1117-462b-972a-edf3b48324fc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.833806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.834484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.834767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.835038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.843529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:48.844043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.845900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:48.858304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:48.859868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:14:48.861034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:48.864246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 290/585] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:48.864863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.878623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413c621-e228-43e4-8e57-56ae1302a48c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:48.915422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:48.916052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:48.916867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:48.917140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:48.927733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:48.929188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:48.943969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:48.950843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:48.951743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:48.951743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:48.951743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:48.951743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 296/586] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:48.951743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:48.981935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87457c-67c1-4176-9fa8-15694be8fa1d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.024630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.025329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.025718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.026126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.027040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:49.027411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.028926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.050195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 291/587] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Sep 3 01:14:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:49.050195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.056699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.065010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459b4d2-eedb-4115-b6f0-1e85c198dcb3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.089687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.103328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.104228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.104723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.105052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/1749220d9e9547eba7ea035508c365bd/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.120990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=1749220d9e9547eba7ea035508c365bd, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.122498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.159901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa858cc0-f0e7-4a85-a1ef-0c606d83d969 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.193862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.194504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.194786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.195058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.195783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:49.196087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.197636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.210856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 297/588] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:48 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/1749220d9e9547eba7ea035508c365bd/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:49.212011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.222699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 292/589] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:49.222699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.295741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.338861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52036a4-fdb7-4331-bd1f-24746907a0ea None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.367520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85b418-3721-4c48-b93b-663e71fa59be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.387365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.388402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.388953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.389485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.404183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.405897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.407218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.407828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.408107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.408377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.409311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:49.409616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.411478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:49.424061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:49.425011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:49.425885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:49.425885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:49.425885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:49.426053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 298/590] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:49.426053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.427780 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 293/591] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:49.433950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.476876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.551385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de09faf-bde2-4e43-a39f-0e98933b71c7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.566608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33641e-e2d2-4f90-9de6-c02bd4a30e00 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.585524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.596637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.597683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.598114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.598399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.605755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.606793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.607256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.607706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.612362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.614182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.617779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.619611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:49.630723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:49.631700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:49.632579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:49.632579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:49.632579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:49.634543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:49.635988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 01:14:49.636925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:49.637446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 299/592] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:49.638041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.640012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 294/593] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:49.645663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.754837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5d71a-6a3f-4cf8-9997-d4260f6cc7a7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.759527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.776023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97844-3a3c-4dba-8061-474108cd83b6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.798556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.799534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.800036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.800507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.803358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:49.803897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.805514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.820364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 295/594] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:49.821620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.832565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.833206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.833490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.833770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.834467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:49.834775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.836703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.854454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 300/595] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:49.855628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:49.866469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:49.937499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc95497-d9fe-4b06-8460-14c5810ab3d2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:49.975734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:49.977052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:49.977644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:49.978306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:49.979516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:49.979922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:49.981790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:49.997227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63149bd5-0deb-418c-b365-2f9f75be6a29 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.003286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 296/596] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:50.008771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.040794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.042377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.042741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.043070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.051962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.053427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:50.069068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:50.070004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 301/597] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:50.076448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.081731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.125616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.133135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e07291-d33c-4d59-9d26-abb4bd74658b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.172858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.173940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.174480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.175047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.183826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.186831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:50.199585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:50.200672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:50.201782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:50.203430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 297/598] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:50.204463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.209294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b1cb4-3ce6-4e2f-975f-8bf0a319a66a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.255832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.256995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.257894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.258317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.273327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.276021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:50.295743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:50.296919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:50.298024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:50.298024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:50.298024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:50.300501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 302/599] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:50.301469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.329782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773113e2-98d7-4e53-8a6b-a7f9a53faa23 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.369615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.379545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.380606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.382417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.382417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.382417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:50.383342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.384859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.406729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 298/600] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:50.406729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.424105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7985d-16ed-4968-bfeb-1a2f82d19fff None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.424747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.462820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.463933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.464447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.464946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.465841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:50.466364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.468181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.482225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 303/601] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:50.483443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.529991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c033ff7-57b2-449d-b6f6-8e71c89e8104 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.566947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.568133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.568639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.569121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.569982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:50.570525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.572099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.585294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 299/602] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:50.586424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.593368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21b3e4-67ba-476f-a0af-c4b7b701d16f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.610892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.627869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237cd42-7083-4f58-8a5c-ead973fe114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.628536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237cd42-7083-4f58-8a5c-ead973fe114f None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.628822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237cd42-7083-4f58-8a5c-ead973fe114f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.629101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237cd42-7083-4f58-8a5c-ead973fe114f None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/edb970ecb499474f87e02b9ee49c8c8a/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.643636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237cd42-7083-4f58-8a5c-ead973fe114f None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=edb970ecb499474f87e02b9ee49c8c8a, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.645103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237cd42-7083-4f58-8a5c-ead973fe114f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.669364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.693645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336d78f-95a7-4a27-858a-d0e5e7b3b53f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.714385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 304/603] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:50 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/edb970ecb499474f87e02b9ee49c8c8a/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:50.715914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451f67-35cf-4448-821f-5fc653550d55 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.716335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451f67-35cf-4448-821f-5fc653550d55 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.716756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451f67-35cf-4448-821f-5fc653550d55 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.718332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 305/604] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:50.719015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451f67-35cf-4448-821f-5fc653550d55 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.744118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711ec03-a6ff-4211-ae16-bfe8eec4dd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.745232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711ec03-a6ff-4211-ae16-bfe8eec4dd3e None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.745515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711ec03-a6ff-4211-ae16-bfe8eec4dd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.745783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711ec03-a6ff-4211-ae16-bfe8eec4dd3e None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/cbc9f7e49e5946f48df125d4efd80dfc/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.767126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711ec03-a6ff-4211-ae16-bfe8eec4dd3e None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=cbc9f7e49e5946f48df125d4efd80dfc, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.768661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711ec03-a6ff-4211-ae16-bfe8eec4dd3e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.857006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451f67-35cf-4448-821f-5fc653550d55 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:50.864883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 300/605] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:50 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/cbc9f7e49e5946f48df125d4efd80dfc/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:50.866582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.867016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.867431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.894127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:50.895911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:50.908021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:50.908571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:50.908853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:50.909169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:50.920259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:50.922395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:50.938869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:50.940039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:50.941240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 306/606] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:50.947191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:50.987161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:50.989145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 301/607] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:51.083568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2f47c-03fb-4d67-ad28-9d03b351bebd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:51.121541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:51.122146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.122448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.122725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.123422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:51.123720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:51.125147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:51.138217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 307/608] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:51.144207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:51.206709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:51.249653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca0b4-0bd4-4d69-a38d-a558d323e900 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:51.285561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692ceb1-858f-4a09-920a-bee9f7c57be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:51.286320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692ceb1-858f-4a09-920a-bee9f7c57be8 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.286691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692ceb1-858f-4a09-920a-bee9f7c57be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.287032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692ceb1-858f-4a09-920a-bee9f7c57be8 None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/d50713b3e4864430a69d0e00068257de/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.296782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:51.313546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692ceb1-858f-4a09-920a-bee9f7c57be8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=d50713b3e4864430a69d0e00068257de, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:51.316758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692ceb1-858f-4a09-920a-bee9f7c57be8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:51.399841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 302/609] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:51 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/d50713b3e4864430a69d0e00068257de/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:51.407793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:51.408869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772f0f-1609-4c61-817e-760de7f978b9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.409424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772f0f-1609-4c61-817e-760de7f978b9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.409995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772f0f-1609-4c61-817e-760de7f978b9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.412074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 303/610] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:51.425427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd704-19ef-463c-8381-c46798880924 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.432763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd704-19ef-463c-8381-c46798880924 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.432763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd704-19ef-463c-8381-c46798880924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.432964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf699b3-f815-437a-a54f-dd74d2ee1e04 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:51.459420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d4acd704-19ef-463c-8381-c46798880924 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:51.460915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d4acd704-19ef-463c-8381-c46798880924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:51.476607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:51.477269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.477559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.477835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.496644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:51.501681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:51.502855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:51.503921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:51.507753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 308/611] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:51.509021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:51.555110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 304/612] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:51.641094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583d4e7-cff8-4897-9e7c-5c4a0bf6dd6c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:51.677608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:51.678141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.678448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.678787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.679455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:51.679755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:51.681231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:51.690274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 309/613] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:51.695689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd704-19ef-463c-8381-c46798880924 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:51.734183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:51.806921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd704-19ef-463c-8381-c46798880924 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:51.853968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:51.854815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:51.855141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:51.855416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:51.870208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:51.872659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:51.893718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:51.894748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:51.895895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:51.895895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:51.895895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:51.895895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 305/614] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:51.900857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:51.911160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:51.918411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:52.015307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ebe44-6c9f-4bdc-bd20-5255a5dc7eb0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.049442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:52.052557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.053288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.053634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.053970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.054729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:52.056201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.057930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.074006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7b3d2-e944-4a7e-ac21-8e40a04f5b22 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.087767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 310/615] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:52.094327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.147705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.151448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.151830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.153340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.173923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:52.180133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:52.181066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:52.182481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:52.186864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 306/616] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:52.187961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.212226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:52.245132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9c34e-8fc8-46af-849b-0be5b2ecf99a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.281015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.281555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.281831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.282097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.290508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.292098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:52.303735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:52.304693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:52.305639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 311/617] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:52.310943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.422961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffe65e-92a5-4e3c-8592-8a5c2377f411 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.454129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:52.462055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.462752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.463167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.463476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.464383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:52.464673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.466217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.476787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df2f13-843a-4ad3-a89a-19b2d2964966 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.486578 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 312/618] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:52.492405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.545048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.545597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.545878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.546156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.546853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:52.547160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.548563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.569829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 307/619] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:52.575486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.600810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:52.615399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9554eb8-c406-46a7-8c6e-15106a317fdc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.658491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.659242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.662360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.662360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.675860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.679902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:52.709180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:52.710180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:52.711123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:52.711123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:52.711123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:52.711123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 313/620] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:52.720656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.806122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:52.825348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fce84-e60c-4365-926b-fb215467c009 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.887697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.890478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.890767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.891041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.913352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.916710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.931553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d5905-fbbd-4547-a5e6-7bd519ea4516 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:52.951935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:52.952959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:52.953844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:52.953844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:52.953844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:52.958781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 308/621] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:52.964803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:52.973910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:52.974570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:52.974852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:52.975158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:52.975828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:52.976129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:52.977562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:52.991139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 314/622] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:52.996859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.083336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.114469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.118581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050829e-47b8-4574-a09d-1a9db7cf98c2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.130779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d020799-ab0e-445c-8f39-354e597137f1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.162483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.163634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.164090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.164557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.167311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.168113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.168549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.168791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.169568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:53.169941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:53.171518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1de60-acb4-4f88-abf4-4038239bfe2b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:53.173760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:53.175827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:53.184289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 309/623] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:53.185692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0360a-7db9-4b52-8ac8-d317a41caf1b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.186048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0360a-7db9-4b52-8ac8-d317a41caf1b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.186434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0360a-7db9-4b52-8ac8-d317a41caf1b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.191785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 310/624] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:53.191785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0360a-7db9-4b52-8ac8-d317a41caf1b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-449cda97-32b2-4ba8-957d-b00ad313b27b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:53.195175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:53.196055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:53.196957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:53.200273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 315/625] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:53.201626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.202072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.202532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.224437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:53.226509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:53.308398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 316/626] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:53.314782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.325472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0360a-7db9-4b52-8ac8-d317a41caf1b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.346724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.361894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.362774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.363825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.363825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.372378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:53.373939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-43709a2c-6a60-427b-8bff-e48d76fc8a79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:53.386130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:53.387310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:14:53.388486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:53.391018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 311/627] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:14:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:53.392565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460fc7a-eed1-4b44-ac90-ee768e53b0f9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.392948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460fc7a-eed1-4b44-ac90-ee768e53b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.393318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460fc7a-eed1-4b44-ac90-ee768e53b0f9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.398370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 312/628] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:53.398370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460fc7a-eed1-4b44-ac90-ee768e53b0f9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.426409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.438111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8e2a-0de9-4129-940a-fe9d23ce003c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.476266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.477137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.477561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.477971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.479479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:53.479946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:53.481625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bfc29-7215-4af6-9665-affa6f078f72 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:53.495037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 317/629] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:53.496553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.496968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.497391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.513782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460fc7a-eed1-4b44-ac90-ee768e53b0f9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.517206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:53.518631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:53.551059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.552163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.552554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.553077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.554373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:53.554787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:53.556338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:53.579282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 313/630] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:14:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:53.580212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.595909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 318/631] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:53.597080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.624787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.703338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6d73a-2ced-4614-b1ff-52905b2d1c33 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.704214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.725203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b67db3-c770-4f2d-abbe-6606f5128138 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.748747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.749696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.749977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.750246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/0711b98a2c214b7cb807f2d211b628c7/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.764773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.765747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.766275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.766784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.767928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=0711b98a2c214b7cb807f2d211b628c7, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:53.769984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:53.779861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:53.785301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:53.788951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 319/632] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:53.789833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.855054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 314/633] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:14:53 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/0711b98a2c214b7cb807f2d211b628c7/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:14:53.857119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:53.929638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70b751-f02a-4d5d-8d13-d1745f8548be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.962777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:53.973406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472511c3-b4e3-485d-87e9-bc68eb8aeb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:53.974024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472511c3-b4e3-485d-87e9-bc68eb8aeb96 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:53.974306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472511c3-b4e3-485d-87e9-bc68eb8aeb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:53.974630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472511c3-b4e3-485d-87e9-bc68eb8aeb96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:53.976741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fa1ad-a26e-4f56-8d3d-b6d81e790581 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:53.987521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.019014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.019014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.019347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.019582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.020503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:54.021055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.022233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.030975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 315/634] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:54.031840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.059581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472511c3-b4e3-485d-87e9-bc68eb8aeb96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.061130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472511c3-b4e3-485d-87e9-bc68eb8aeb96 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.137163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216182cf-a6bd-4b0c-8066-91aad37c778c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.172215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.172839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.173105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.173371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.178429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 320/635] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:54.179859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545cc87-8f3f-4e92-89d3-0eb4f776e4c2 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.180295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545cc87-8f3f-4e92-89d3-0eb4f776e4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.180703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545cc87-8f3f-4e92-89d3-0eb4f776e4c2 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.183167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 321/636] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:54.184072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545cc87-8f3f-4e92-89d3-0eb4f776e4c2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.185967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:54.192472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:54.193405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:54.194256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 316/637] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:54.195505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.211495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.232746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.301577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545cc87-8f3f-4e92-89d3-0eb4f776e4c2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.303995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cd910-6c99-45a2-b773-e89a66a93c04 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.340514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.340751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.341393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.341672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.341942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.342161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.342651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.343101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.353136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.354713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.355336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.356974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7f2bf1d8-0c3b-4fda-ab38-db9c3957a723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:54.368973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:54.370589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:54.371761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:54.371761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:54.371761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:54.375133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 317/638] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:54.375709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:54.376891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:54.377984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:54.377984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:54.377984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:54.378203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.378203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.378203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.379661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 322/639] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:54.380613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.400921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:54.402069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:54.460597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.471636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 318/640] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:54.472474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.508022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.521114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88037f6-7183-4283-9ad3-2ec96e00bc85 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.568535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.569299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.569673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.570066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.570873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:54.571317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.572864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.581656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 323/641] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:54.582582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.585613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41211c-4a83-4b28-94f0-4a3010a3a0d8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.624438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.625226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.625596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.625962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.626837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:54.627233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.629275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.641237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 319/642] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:54.646605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.700662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872047a3-6050-4e87-a99f-7ee8448c70a6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.715092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.736517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.737313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.738281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.738281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.738875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:54.739287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.740763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.757654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 324/643] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:54.757654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.773549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:54.775262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e36ef-565a-4a1f-b83a-680ae3e3c721 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.814647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.815286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.815563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.815831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.829197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.830799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:54.845997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:54.847153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:54.848064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:54.848064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:54.848064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:54.848064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 320/644] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:54.851495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.871181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80d8e6-e7cd-4e07-882e-6cbed8f49c99 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.907986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:54.908629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:54.908914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:54.909191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:54.916766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:54.918358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:54.929662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:54.930598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 325/645] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:54.932634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:54.973959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef942c-ba9b-4641-aa7d-d5e2baff5b71 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:54.976913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.009555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.010748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.011032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.011300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.018115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.022301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:55.027759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:55.028887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:55.029928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:55.035903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 321/646] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:55.035903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.048188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458c60-2d34-4eb9-bf5b-be52fa5c139a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.081778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.082696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.083143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.083553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.091848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.093433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:55.104537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:55.105498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 326/647] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:55.106977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.144212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1d966-be6a-4b48-a827-a0b2086ec990 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.178990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.179635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.179871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.180149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.180791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:55.181101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.183067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.199076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 322/648] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:55.199076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.210557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c025a3-6835-4bf2-a652-56306403d8ff None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.223828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.253004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.253004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.253004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.253004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.253387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:55.253387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.253387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.253387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.268767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 327/649] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:55.268767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.307578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02556e11-abf2-4ed3-947f-2ae10ce849e1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.342883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.343492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.343770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.344038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.344696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:55.344998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.346384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.359148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 323/650] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:55.359148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.380779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105c59b-d05a-4b32-8ed9-34b69b002d05 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.417553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.418332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.418697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.418967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.419689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:55.419988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.421664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.448442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 328/651] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:55.448442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.453847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.470787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fa64a-af94-47e3-b279-0854202dd03e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.503145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.507763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.508476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.508828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.509191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.517704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.519230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:55.531235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:55.532390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 03 01:14:55.533522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:55.537293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 324/652] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:55.538107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.556141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937f43-c41d-4dd6-b628-e59e97d521c7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.592163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.592940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.593318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.593689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.594610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=1749220d9e9547eba7ea035508c365bd) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:55.595130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.596665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.604184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 329/653] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/role_assignments?user.id=1749220d9e9547eba7ea035508c365bd&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:55.604888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.673151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69972dc-22ff-428c-b366-7398cf78e33a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.712064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.712924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.713391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.713671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.728947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.730948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.740015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.741510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fa85d-7ee8-45ff-802d-81d79d25d6c4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:55.755369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:55.756349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:55.757231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:55.757231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:55.757231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:55.757231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 325/654] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:55.757231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.772232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:55.792779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a249aae-6924-42c5-98c2-3d4d6d710e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.793494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a249aae-6924-42c5-98c2-3d4d6d710e9d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.793847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a249aae-6924-42c5-98c2-3d4d6d710e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.794195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a249aae-6924-42c5-98c2-3d4d6d710e9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.877167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c50f3a-ab43-446b-9037-cc5289e64b45 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:55.883460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a249aae-6924-42c5-98c2-3d4d6d710e9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.884998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a249aae-6924-42c5-98c2-3d4d6d710e9d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.915295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:55.915988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:55.916986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:55.917326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:55.918013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:55.918319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:55.919813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:55.937458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 326/655] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:55.938304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:55.996743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.008261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 330/656] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:56.009559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fc766-6a0a-4a80-a95b-6ab13be94804 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.010459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fc766-6a0a-4a80-a95b-6ab13be94804 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.010739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171fc766-6a0a-4a80-a95b-6ab13be94804 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.012186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 331/657] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:56.016282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fc766-6a0a-4a80-a95b-6ab13be94804 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.039813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.056298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c97ef-ab31-4266-9f6d-8a63e9c8d098 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.094474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.095148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.095433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.095705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.096346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:56.096647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.098023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.111777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 327/658] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:56.112615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.136958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171fc766-6a0a-4a80-a95b-6ab13be94804 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.179390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.180971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.182282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.183069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.218494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4b4d-4106-497a-8662-c3c64f67d8c1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.234855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.254316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.254981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.255259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.255530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.260787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.262531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.265811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.267344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.281396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5611513f-145e-4f34-b431-a6593c97878d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:56.282462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:56.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 328/659] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.289113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.304303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:56.305503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:56.378202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 329/660] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:56.379435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.384995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 332/661] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:56.384995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.502907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50f67a-3dca-4ee6-8569-20520baa4020 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.507471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2940f-9e65-4797-bb6d-66805c58a261 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.526736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.527205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.548271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.548885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.549162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.549430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.550074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:56.550390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.551768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.552752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.553610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.554129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.554646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.562974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.564683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.568630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 333/662] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:56.569514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:56.578278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:56.585706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 330/663] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:56.586898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.681094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aaeda-9a4e-482a-8b6e-e90bcccb2941 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.689733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fe5f8-fdd5-47f3-b94e-171e458ea489 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.716674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.717563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.717841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.718120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.728103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.729346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.729598 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.730043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.730142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.731610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.736932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.738377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:56.745964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:56.746985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:56.747904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:56.747904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:56.747904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:56.747904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 334/664] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:56.748730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:56.752416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:56.753300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:56.754466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 331/665] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:56.755084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.774387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.785279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:56.864458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432bbe-cad1-450b-b452-ae70b533c4db None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.871226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152b2e-bdaa-41d6-b8e6-4d19c88e8d3e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:56.901670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.902301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.902612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.902883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.907417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:56.908217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:56.908813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:56.909277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:56.910171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:56.910681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:56.912319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:56.913149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:56.918452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:56.919373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:56.920209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 332/666] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:56.921035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:56.934065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 335/667] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:56.940060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.013554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.026386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.035227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31d412-fc7f-4b44-b326-0714e0d52ace None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.041656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82674581-bb9e-4a17-81d8-b527c64b487e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.070817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.071429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.071711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.072014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.073524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:57.073844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.075245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.077329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.077937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.078216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.078509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.079200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:57.079501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.080920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.089617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 333/668] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:57.090720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.098382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 336/669] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:57.098382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.197900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e735c5-3d08-42bb-bbf4-7a84e5905ed7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.204212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da290a-d47f-49fb-ab25-9a860f3bcdf3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.232455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.233069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.233334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.233600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.234257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:57.234581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.235948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.240913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.242247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.242896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.243175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.243446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.249347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 334/670] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:57.249347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.265379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.324388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.325876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.354501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0079411-ac88-47e5-bf79-b1c36b050fc3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.390893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.391519 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.391792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.392058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.399057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.401075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:57.412303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:57.413217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:57.414085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:57.418221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 335/671] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:57.419091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.435331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 337/672] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:57.436389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.484042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.485102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.528376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab5d08-e0a6-425a-b43a-019b23f8694e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.544583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1f6c7-1822-4d1e-9505-cddc175196f3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.563643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.564251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.564527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.564795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.565517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=edb970ecb499474f87e02b9ee49c8c8a) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:57.565817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.567299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.573181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 336/673] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/role_assignments?user.id=edb970ecb499474f87e02b9ee49c8c8a&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:57.573968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.594907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.595903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.596196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.596469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.613991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.615933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:57.636505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:57.637741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:57.638882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:57.638882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:57.638882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:57.638882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 338/674] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:57.639180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.698102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86940f-cfee-4f1d-b85a-dd0f405ba94e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.726534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.741269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.741890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.742169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.742505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.743208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:57.743514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.746024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.758906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 337/675] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:57.759759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.763137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:57.767102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa525a05-8c3f-4f83-8327-e1b5bb7816a5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.806086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.807404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.808073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.808349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.819416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.823036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3c07cf13-14f4-42eb-a498-683d702181f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:57.841139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:57.842116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 339/676] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dc39b-4cbb-431d-a041-a8c8321e5c2a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dc39b-4cbb-431d-a041-a8c8321e5c2a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.848237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dc39b-4cbb-431d-a041-a8c8321e5c2a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.849761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 340/677] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:57.850390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dc39b-4cbb-431d-a041-a8c8321e5c2a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.882504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f70a2-b0d0-4c5a-a634-253420a3b092 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.926679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:57.927296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:57.927612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:57.927853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:57.928507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:57.928812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:57.930214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:57.942785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 338/678] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:57.943455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:57.967625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dc39b-4cbb-431d-a041-a8c8321e5c2a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:57.995909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.003768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.004426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.004706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.004975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.005854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=cbc9f7e49e5946f48df125d4efd80dfc) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:58.006416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.007687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb317b6-b86e-4d81-b1ca-9ba9bddd2a6d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.014523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 341/679] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/role_assignments?user.id=cbc9f7e49e5946f48df125d4efd80dfc&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:58.016098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.016604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.017070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.040878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:14:58.046441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:14:58.059799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.086682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5053fe5-e03e-48f9-812d-b0d43ddcab0e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.137986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 342/680] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:14:58.138884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.142803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.143526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.143812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.144090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.144767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:58.145078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.146824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.169808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 339/681] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:58.183808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.261108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09f57-53ed-40a2-a9ca-0c90cadcbe99 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.303676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.304763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.305249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.305720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.315820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.317640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:14:58.330587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:58.331628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:14:58.332618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:58.333192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6bc0e-cda9-4b21-90ff-9861bec0f3fe None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.335081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.336443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 343/682] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:58.342674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.347413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.377231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.378409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.385607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.385607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.394642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.394642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:58.408865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:58.409825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:58.410767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 340/683] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:58.416948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.474599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c63aa3-9e3c-43b4-af77-afb0c92099d1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.521996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.522647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.522932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.523206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.523948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:58.524248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.525728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.544392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c7e5-26b4-4003-b173-b18be84645d8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.552946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 344/684] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:58.553856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.589108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.589818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.590102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.590404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.591114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:58.591420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.592943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.605735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 341/685] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:14:58.611180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.634760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.637816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.670582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ba9c6-6396-4303-9e1f-6f30adb4fe28 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.714256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.715253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.715729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.716208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.716864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd827e-92b9-45ec-9528-0de84897a419 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.727494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:14:58.733830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:58.734829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:14:58.735664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:58.738445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 345/686] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:58.739554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.759086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.760300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.760580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.760879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.840220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.841712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.868330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.869463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13b223-6c1e-4328-9d0e-af481085dc40 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:58.898227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:58.913886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:58.915327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:58.915903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:58.916489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:58.917480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=d50713b3e4864430a69d0e00068257de) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:58.918178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:58.920317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:58.928506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 346/687] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/role_assignments?user.id=d50713b3e4864430a69d0e00068257de&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:58.930056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:58.984557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 342/688] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:14:58.988901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.140156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc75bb0-47b2-4c93-bec6-c270dd1bcf73 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.169814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8679-d30d-44b5-b7fb-228a88dcf9c8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.188293 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.188967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.189398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.189773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.193781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:59.194230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.195755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:59.205100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:59.207383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 347/689] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:59.214943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.224313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.236922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.236922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.236922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.239462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.241489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:59.280749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:59.281879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:59.282875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:59.282875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:59.282875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:59.282875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 343/690] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:59.289295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.297394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:59.384682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7819daa3-4aca-4543-9db5-a9a273cc402b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.424136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c44e5-0c10-434f-9586-28687f01f091 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.459366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.460693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.461199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.461989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.478033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.478781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.479148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.479427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.480073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:59.480369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.481870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:59.485841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.488395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:14:59.513544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:59.515039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:14:59.515946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:14:59.515946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:14:59.515946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:14:59.517500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 344/691] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:59.522446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 348/692] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:14:59.523475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.528081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.596620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:59.676262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faba81-8fe7-4893-92e7-ab74d33be01c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.679668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:14:59.718989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.720163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.720636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.721567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.722794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:14:59.723956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.726160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:59.736405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a226-0b52-4f93-8ec9-877b5d2b93dc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.749436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 349/693] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:14:59.756554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.814941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.816056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.816701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.817176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.834784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.837279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d2f428c1-d004-449e-abcf-6722e15ab00b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:14:59.855398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:14:59.856352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:14:59.857235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:14:59.859232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 345/694] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:14:59.861819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c0bf1-bbd3-4fd8-8bcd-fb440cb668f3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.862291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c0bf1-bbd3-4fd8-8bcd-fb440cb668f3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.862735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c0bf1-bbd3-4fd8-8bcd-fb440cb668f3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.868502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 346/695] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:14:59.871374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c0bf1-bbd3-4fd8-8bcd-fb440cb668f3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:14:59.910137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952eef9c-8c2b-4aab-a8b9-d631ccebdfab None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:14:59.961767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:14:59.962634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:14:59.963732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:14:59.964409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:14:59.979954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:14:59.981776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0f283cc7-4d3a-4872-83b5-189543f8dfd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:00.000824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:00.001855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:00.002836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 350/696] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:14:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:00.011068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.011068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.011068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.026068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.036762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c0bf1-bbd3-4fd8-8bcd-fb440cb668f3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.046072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:00.049202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:00.078024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.083562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.084927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.085764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.091277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.092579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:00.093095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.095016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.111259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 347/697] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:00.113156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.152261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 351/698] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:15:00.153354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.234738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0708d-177f-4e24-92a4-93714233b174 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.275886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.276780 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.277216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.277637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.278558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:00.279014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.280656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.305456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 348/699] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:00.307165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.323427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.343240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0d2fc-30dd-4008-af45-33f42ffcabf2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.392461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.393086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.393368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.393638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.394403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=0711b98a2c214b7cb807f2d211b628c7) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:00.394718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.396432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.415199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 352/700] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/role_assignments?user.id=0711b98a2c214b7cb807f2d211b628c7&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:00.439978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d337-b796-4c03-a35c-3eb908bddba4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.482753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.483662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.484110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.484540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.486383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.546786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.567573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.569334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.586858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.656920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714580-8e8a-46c2-b977-bc3677b50e6d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.695757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.696397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.696685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.696988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.697601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.699037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.701078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 349/701] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 03 01:15:00.702522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:00.717458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:00.718587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:00.719522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 353/702] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:00.726331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.835822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.848405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5bd1e-25c4-4d48-a83d-6629b652e123 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.859821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4425d-51a4-4a25-aad0-407ddfa4e58f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:00.866601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:00.893184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.894073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.894374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.894656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.900586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:00.901432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.901771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.902089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.902754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:00.903112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.904703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67f9f7-7093-4f90-81d9-4a04fd4c46e6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.906414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:00.908232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:00.916482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 354/703] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:00.917897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660af310-57a4-48f4-bfe1-25fbda653dee None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.918188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660af310-57a4-48f4-bfe1-25fbda653dee None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.918567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660af310-57a4-48f4-bfe1-25fbda653dee None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.921277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 355/704] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:00.924287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:00.925212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:00.927009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:00.927009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:00.927009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:00.927009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 350/705] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:00.927539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:00.927539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:00.927539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:00.935095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:00.953212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:00.954670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:01.047157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 356/706] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:15:01.048079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:01.082294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee719334-95f1-4f81-bfe2-07307291b3e0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.135410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.136817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.137101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.138246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.139732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:01.140041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.141455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.153037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 351/707] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Sep 3 01:15:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:01.182028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:01.222690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bb7757-fde1-414e-85b9-b330694694b0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.276217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.276855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.277136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.277409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.278072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:01.278394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.279804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.299779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 357/708] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:01.305349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:01.350451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:01.429227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:01.429692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75b093-de21-444f-b02b-1c9d2a3f6504 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.483110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.483895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.484193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.484476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.495642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.497129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e56a9d17-f264-4c85-a72d-5d8fc7a78b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:01.512994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:01.513949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:01.514903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:01.518421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 352/709] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:01.519856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2878a-f99e-4247-87ea-8daf814883c2 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.520626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2878a-f99e-4247-87ea-8daf814883c2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.521102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2878a-f99e-4247-87ea-8daf814883c2 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.523134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 353/710] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:01.527729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2878a-f99e-4247-87ea-8daf814883c2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:01.580498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acd3bc-4d8d-4dec-a377-aa0cd9777056 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.617062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:01.623288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.624537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.625102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.625613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.630725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:01.638800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.640534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.643106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2878a-f99e-4247-87ea-8daf814883c2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0d83014b-f60a-4419-990d-62eabb03d1d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:01.657276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:01.658309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:01.659253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:01.659253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:01.659253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:01.666480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 358/711] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:01.668364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.668856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.669332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.680862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.681508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.681789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.682061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.682831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:01.683140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.684592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.691545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:01.694104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:01.704284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 354/712] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:01.704284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:01.783531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 359/713] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:15:01.784402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:01.892310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655f1db-e445-4233-955f-ba18a7d30a26 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.913555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5c694-8a82-49cd-b894-0b80e48ddb4d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:01.926953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:01.955479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.956548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.958166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.959520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.961014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=1749220d9e9547eba7ea035508c365bd) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:01.962361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.971006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.976286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:01.980241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:01.980241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:01.980241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:01.980241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:01.980241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:01.982637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:01.990506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 360/714] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:01 2022] GET /identity/v3/role_assignments?user.id=1749220d9e9547eba7ea035508c365bd&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:02.005660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 355/715] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:02.010560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:02.147277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fc5de-6781-42fb-953d-47e4eba39b0d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:02.195995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:02.198173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:02.198917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:02.199424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:02.200365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:02.202187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:02.214267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:02.229062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:02.309564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 361/716] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:15:02 2022] POST /identity/v3/users => generated 254 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:02.335143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39be2a-893c-4edf-b6ce-e9ce55bb4cc8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:02.373992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:02.374658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:02.374938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:02.375208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] PATH_INFO: `/v3/services/network` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:02.375796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:02.377185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:02.389302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:02.390390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:02.391640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 356/717] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:02 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:02.399569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:02.482406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:02.575679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f4c69-778e-46f2-9491-38120dfe2b50 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:02.620620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:02.621243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:02.621531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:02.621812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:02.622481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:02.622789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:02.624168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f405c13-8815-4322-baec-f62679e5ceb1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:02.633662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 362/718] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:02.635779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:02.715981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeafff-6997-414c-8ef1-885822f6e7e4 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:02.716298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeafff-6997-414c-8ef1-885822f6e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:02.716480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeafff-6997-414c-8ef1-885822f6e7e4 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:02.717813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 363/719] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:02.720842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:02.722974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:02.723536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:02.723536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:02.743186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:02.745791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:02.747433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c74429-5671-4a28-926c-69a2b2bd1e72 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:02.810871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:02.811541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:02.811880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:02.812162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:02.812825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:02.813134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:02.814909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:02.828620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 357/720] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:02.877267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 364/721] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:15:03.211915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:03.355075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcc3fe-7e5d-4f7f-a966-6ba3ecb020b8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:03.408713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:03.409351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:03.409632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:03.409903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:03.421187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:03.430414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:03.435943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:03.447646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:03.448751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 358/722] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:03.449470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:03.485054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:03.578771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c1e4-2109-4b8c-a1a8-0d6b17117f3a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:03.615197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:03.615811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:03.616097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:03.616373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:03.617044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:03.617353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:03.619314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852d4c5-609e-48c5-adce-2bde4386ce7d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:03.628723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 365/723] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:03.633966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:03.686365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93096-f163-4618-9205-05e113cd9469 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:03.686714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93096-f163-4618-9205-05e113cd9469 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:03.686932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e93096-f163-4618-9205-05e113cd9469 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:03.688529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 366/724] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:03.693665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:03.693954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:03.694232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:03.715147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:03.716349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:03.817441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b797bc-21d2-4680-a249-76a409453195 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:03.859143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 367/725] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:15:03.887799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:03.888467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:03.888747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:03.889050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:03.910119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:03.911649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:03.935389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:03.936388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:03.937886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:03.937886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:03.937886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:03.937886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 359/726] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:03.942207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:03.970513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:04.061762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658a8af-c2ad-4eb5-9bc0-29977807e346 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:04.096200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:04.096808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.097091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.097365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.098011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:04.098313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:04.099718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:04.115133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 368/727] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:04.121312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:04.185057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:04.248591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fed65-3647-4732-99ff-00eb70d06ce9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:04.285354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:04.285994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.286283 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.286583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.294928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:04.296413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bd7bd2af-9a55-45c4-806b-eed875905518 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:04.308698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:04.309647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:04.310625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 360/728] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:04.315584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:04.325453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bef14d-4aa7-4d46-8d34-df265ffc06af None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.325745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bef14d-4aa7-4d46-8d34-df265ffc06af None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.326026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bef14d-4aa7-4d46-8d34-df265ffc06af None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.327535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 361/729] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:04.333129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.333414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.333689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.356572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:04.357798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:04.441366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:04.449445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 362/730] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 03 01:15:04.453611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:04.480115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa5f00-bd56-46f9-b80d-d1ebb08dc937 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:04.539908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:04.550971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:04.551668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.551951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.552223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.552974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:04.553279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:04.554745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:04.593218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 369/731] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Sep 3 01:15:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:04.598722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:04.665119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a7c27-ed72-4d8d-a6f4-c04b751742dd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:04.729735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be350b-0236-4c56-af16-8bb5629de9bd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:04.758759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:04.760356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.760854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.761338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.762008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:04.763455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:04.767257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9757-40c3-4941-aec3-f34795f5317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:04.768048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9757-40c3-4941-aec3-f34795f5317d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:04.768343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9757-40c3-4941-aec3-f34795f5317d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:04.768744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9757-40c3-4941-aec3-f34795f5317d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:04.778734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:04.848140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9757-40c3-4941-aec3-f34795f5317d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:04.849524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9757-40c3-4941-aec3-f34795f5317d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:04.862200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 363/732] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:04 2022] POST /identity/v3/services => generated 251 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:04.863635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.002417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 370/733] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 03 01:15:05.003850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2e441-8412-4315-8c93-80ebc0a1c5c9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.004568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2e441-8412-4315-8c93-80ebc0a1c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.004568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2e441-8412-4315-8c93-80ebc0a1c5c9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.006127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 371/734] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:05.007643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2e441-8412-4315-8c93-80ebc0a1c5c9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.023354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.031498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fded1f9-9d76-468c-8614-92be0897c2f3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.070432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c31a-dd81-4968-805a-e2d89f3f58bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.071082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c31a-dd81-4968-805a-e2d89f3f58bf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.071368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c31a-dd81-4968-805a-e2d89f3f58bf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.071750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c31a-dd81-4968-805a-e2d89f3f58bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.082913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.083674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c31a-dd81-4968-805a-e2d89f3f58bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2827c31a-dd81-4968-805a-e2d89f3f58bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:05.089668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:05.090943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:05.092008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:05.094505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 364/735] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:05.098262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.098262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.098262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.115601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:05.117099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:05.159851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2e441-8412-4315-8c93-80ebc0a1c5c9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.193418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 365/736] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:15:05.194897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.204305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.205657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.205997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.206386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.218724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:05.220510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:05.244944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:05.246058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:05.247010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:05.247010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:05.247010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:05.250739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 372/737] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:05.251850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.302747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.315820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ad705-b226-46b5-8a7d-943aed088bc3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.353858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.354975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.355444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.355894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.356821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:05.357329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:05.359107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:05.369628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 366/738] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:05.376272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.434236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.444698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f4b45-0f32-4986-bbb4-342b82c05340 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.483728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.485573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.486510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.487145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.488113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:05.488604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:05.491213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc0b90-0876-4e42-b903-8f6a187c6bbd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:05.510693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 373/739] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:05.512055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1a19c-7d29-48cc-8e92-ff5b3db2652a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.512410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1a19c-7d29-48cc-8e92-ff5b3db2652a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.512602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de473b68-f572-43fd-acbc-c218d514cd26 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.512755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1a19c-7d29-48cc-8e92-ff5b3db2652a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.514328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 374/740] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:05.520226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1a19c-7d29-48cc-8e92-ff5b3db2652a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.552173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.553144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.553626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.553904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.565215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:05.566706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:05.575042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9389b01f-65f7-4440-aa06-5fe988c83c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:05.581787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:05.587006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 367/741] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.588490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.604053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:05.605861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:05.637768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1a19c-7d29-48cc-8e92-ff5b3db2652a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.675495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.676122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.676401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.676691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.678605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 368/742] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 03 01:15:05.679593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.695349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:05.697169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:05.709283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:05.710389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:05.711497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:05.713848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 375/743] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:05.719355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.732935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.827590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685882f-12c0-49e6-8b6d-5b33bfefe0b1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.833779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46323964-4c0c-404b-81cd-bdc2f06fd2d4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:05.867696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:05.870792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.871654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.872132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.872598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.873487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:05.873989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:05.874928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:05.875755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:05.876079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:05.876182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:05.882738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:05.889998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 376/744] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:05.890157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:05.891285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:05.896660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:05.897605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:05.898478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 369/745] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:05.899179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:05.984318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.057107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd6971-598d-44bf-b86c-45f2a1bb7b97 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.058759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b406b-7edb-4220-9f0d-1e06b0dbb67a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.094005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.094687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.094969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.095446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.095570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.096100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.096381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.096653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.097349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:06.097704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.099130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.103771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.105202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.111443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 377/746] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:06.116094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:06.117879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:06.118989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:06.119970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:06.123734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 370/747] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:15:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:06.124632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.180060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.226002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db716-aef9-47c4-b575-22d656245e2a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.232189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd38dc-bf15-4d83-b2ec-722e66089a76 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.255901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.263201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.263812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.264160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.264502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.265097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:06.265401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.265629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.266234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.266545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.267288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.267378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.267594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.268919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.276529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 378/748] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:06.277342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.338414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 371/749] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:06 2022] POST /identity/v3/services => generated 240 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:06.338414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.411361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c9527-1729-4df7-b072-7ff50adae4e6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.467825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.468807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.469187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.469668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.470487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=0711b98a2c214b7cb807f2d211b628c7) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:06.470896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.472574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.473405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdf7fe-f61c-44db-bb73-58ddbfb74a39 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.490375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 379/750] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/role_assignments?user.id=0711b98a2c214b7cb807f2d211b628c7&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:06.490375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.495964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.530278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.530936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.531221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.531494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.532241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:06.532548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.534363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.555117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.559469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 372/751] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:06.565813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.620301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85900ae-6c8d-4d14-9705-436c047bbd5f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.668318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.668883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.669269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.669608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.681175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.682672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.697864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79f5fa-61e7-42fa-94bc-622b466a62d9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:06.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:06.700967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:06.701860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:06.701860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:06.701860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:06.705800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 380/752] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:06.708709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.740196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.740843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.741124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.741396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.757452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.758971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:06.785515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:06.786568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:06.787516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:06.787516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:06.787516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:06.787887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 373/753] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:06.794122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.818746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.916722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a543cf9-6e87-40c4-b480-5969d5e46710 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.944707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f2eb4-33e4-47f1-835f-93d135c578cb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:06.947902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:06.956105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.961025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.961025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.961025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.961025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:06.961025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.961025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41fa20-7ca5-44e9-bdcb-ab31795ff0d3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:06.978832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 374/754] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:06.978832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd7955-df8e-46c8-b63d-520f0a55329b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.978832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd7955-df8e-46c8-b63d-520f0a55329b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.978832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd7955-df8e-46c8-b63d-520f0a55329b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.978832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 375/755] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:06.985359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd7955-df8e-46c8-b63d-520f0a55329b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:06.993315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:06.994115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:06.994646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:06.994947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:06.995635 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:06.995945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:06.997491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f654a-575b-4808-a6cf-db744ad8e0fc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.011011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 381/756] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:07.016780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.017150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.017440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.037475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:07.038806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:07.086702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:07.111464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 382/757] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:07.112401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.119634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd7955-df8e-46c8-b63d-520f0a55329b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.187295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.187945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.188230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.188503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.200223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.203154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:07.226311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:07.227376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:07.229051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 376/758] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:07.235144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.294523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:07.305168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd86a9-a465-48ea-9836-c11a318b8118 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.374384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.375030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.375312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.375587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.392031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.393918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:07.415051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:07.418038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:07.420343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:07.426357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 383/759] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:15:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:07.426357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.430803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5379e-b10f-41df-bed9-0da3eae27d16 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.470188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.470902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.471187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.471456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.472147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:07.472449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.473886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.484451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:07.489117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 377/760] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:07.489911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.566795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a042f60-1a02-48ea-b5f5-ec87ad13d0f1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.602883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.604032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.604316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.604659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.605613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:07.605922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.607711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.623471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98a210-fad4-43a2-ae12-020255cb17dd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.629973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 384/761] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Sep 3 01:15:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:07.630832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.647816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:07.672532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.673186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.673673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.673946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.674569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.675995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:07.690167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:07.691294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:07.692152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 378/762] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Sep 3 01:15:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:07.692977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.758870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:07.764846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eadb0e-171f-4b76-95cf-9266fdf03d65 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.801900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8179a-f890-4524-8d5d-a55a712f1353 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:07.807109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.808111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.808602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.809089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/1749220d9e9547eba7ea035508c365bd/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.811864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=1749220d9e9547eba7ea035508c365bd, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.814662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:07.841749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:07.842386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:07.842666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:07.845359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:07.893673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 385/763] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:15:07 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/1749220d9e9547eba7ea035508c365bd/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:15:07.895831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:07.925504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:07.927230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.004418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.043187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c91b79-2306-48a3-b1c4-accbfced53c0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.067523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.088506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.089841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.090409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.091035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.091923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:08.092618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.094513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.101679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 386/764] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Sep 3 01:15:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:08.111964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.152036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 379/765] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:15:08.157486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.239902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1173431-c26a-4fa5-8fdd-9149d790e46a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.283613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ef464-0fea-4ec2-94cd-edbb4d1e981d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.296987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.297541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.297831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.298101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.298778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:08.299083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.301679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.309161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.315351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 387/766] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:08.321394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.352506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.353692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.354165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.354678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.380322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.383178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:08.417632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:08.418731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:08.419636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:08.419636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:08.419636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:08.419636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 380/767] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:08.425580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.466761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.512921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92eca2d-ef4b-4ff0-9f3b-d17d185da3f5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.553873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.554504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.554791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.555070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.555832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] RBAC: Adding query filter params (interface=public, service_id=ba01c6c2e4b148089e1e033d6b11e04d, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:08.556641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.565011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.567101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 388/768] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/endpoints?service_id=ba01c6c2e4b148089e1e033d6b11e04d&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:08.572693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.612236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711038a-b1ca-440b-85d4-c11eabc6bf6e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.650788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.651576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.651967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.652346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.653103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:08.653571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.655112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce44cb6-581a-478d-a3a9-b95da688e7be None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.657420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.668390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 381/769] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:08.669979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654460b7-16a3-446a-9d0b-b98411d0907d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.670270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654460b7-16a3-446a-9d0b-b98411d0907d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.670571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654460b7-16a3-446a-9d0b-b98411d0907d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.671786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 382/770] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:08.672954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f7268-86fb-4786-ad14-8d2f2c05308b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.673295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f7268-86fb-4786-ad14-8d2f2c05308b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.673631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f7268-86fb-4786-ad14-8d2f2c05308b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.675140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 383/771] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:08.676127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f7268-86fb-4786-ad14-8d2f2c05308b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.706164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37484-36b4-4447-a5ab-c8d0b3ab33fa None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.750642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c2a38-e150-4432-8a93-2a26569a801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.751253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2a38-e150-4432-8a93-2a26569a801a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.751530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2a38-e150-4432-8a93-2a26569a801a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.751796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c2a38-e150-4432-8a93-2a26569a801a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.752392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2a38-e150-4432-8a93-2a26569a801a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.754041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c2a38-e150-4432-8a93-2a26569a801a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.765831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.766740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 389/772] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:08.769863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.770902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.771959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.794382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f7268-86fb-4786-ad14-8d2f2c05308b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:08.807715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:08.809739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:08.848389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:08.849424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.850179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.850654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.862624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:08.864411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6f2d4bc0-28b2-45f5-98db-bcdd5c5f08a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:08.879603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:08.880770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:08.881898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:08.887748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 384/773] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:08.887748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:08.887748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:08.887748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:08.905481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:08.906739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:08.911743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 390/774] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:08.912599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:08.981072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:08.996719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 385/775] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:09.035422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa4177-161b-44df-bec5-ca0058fcc1f0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:09.074114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:09.075066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:09.075543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:09.076079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:09.077028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:09.077530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:09.079175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:09.091805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 391/776] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:09.097518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:09.199289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:09.247093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360dc81-ca04-471e-8e46-94372f76bad2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:09.296453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:09.297081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:09.297365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:09.297637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:09.298415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=57efcdf1033744358788130e1c663538) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:09.298729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:09.300182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:09.310716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 386/777] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/role_assignments?user.id=57efcdf1033744358788130e1c663538&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:09.325156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:09.386690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:09.499213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8585c7-a5b8-48a5-bc70-fb6b627c52b6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:09.517814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bedec-163c-45b0-b9dc-e397374671b0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:09.535544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:09.551935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:09.564871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:09.565847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:09.566517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:09.568270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] PATH_INFO: `/v3/services/network` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:09.569180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:09.571681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:09.582577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:09.589397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:09.590033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:09.590546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:09.593394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:09.594385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:09.595223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:09.606391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 387/778] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:09.612014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:09.619507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:09.633267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:09.635044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:09.635872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:09.640051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 392/779] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:09.646374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:09.835605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6160e-a96e-4eec-a194-9b2c79cd9caf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:09.836331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544e02b-e481-49ee-ace3-d42af0f7af92 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:09.877370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:09.877979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:09.878257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:09.878556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:09.879184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] RBAC: Adding query filter params (name=network) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:09.879483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:09.880864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:09.889138 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 388/780] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:09.894108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:09.920354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:09.920973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:09.921250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:09.921727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:09.925660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:09.925973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:09.927559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:09.943646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 393/781] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:09.949612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.002699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:10.030434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:10.076093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d7f7d-807a-44cd-8700-0cc23e2e09fd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.132695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934ea8d-d842-40d2-8246-dd1d9b8dcbea None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.139048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.139662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.139940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.140213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.140850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:10.141147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:10.142548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f77ff-c510-4013-bbec-9c000ed1aae5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:10.151369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 389/782] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:10.152594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafccb9-0514-4625-a1dc-af861288a3e4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.152886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafccb9-0514-4625-a1dc-af861288a3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.157713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafccb9-0514-4625-a1dc-af861288a3e4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.159243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 390/783] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:10.159960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafccb9-0514-4625-a1dc-af861288a3e4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.185895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.186805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.187493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.187767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.202019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:10.204216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0a26de17-dcfb-4aca-8635-3fe86a7919d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:10.221968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:10.223053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:10.223969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:10.223969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:10.223969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:10.223969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 394/784] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:10.224987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.229237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.229237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.249655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:10.251008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:10.280013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafccb9-0514-4625-a1dc-af861288a3e4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.307909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:10.319962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.321789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.321789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.321789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.322123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] RBAC: Adding query filter params (interface=public, service_id=26a537b3d2694d93b5810358187a9ca6, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:10.322507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:10.323952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:10.336059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 391/785] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/endpoints?service_id=26a537b3d2694d93b5810358187a9ca6&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:10.336059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.354404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 395/786] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:10.355524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.383444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:10.472875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdac033-9bfc-49b8-9b82-6b642879b3f5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.506695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd947a-7af2-4acd-9553-633e6eb9f8cf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.545989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.547580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.548038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.549642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.550456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:10.552103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:10.553635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:10.567260 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.568084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.568527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.568941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.569744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:10.571286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:10.580007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 392/787] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:10.582616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 396/788] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:10.587543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.678222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:10.692688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.766786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efaa40-0c10-48d4-85b5-88344737f731 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.817345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:10.817622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.819177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.819177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.819392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.832119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:10.833608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:10.844476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac127f5-cda4-4057-8f46-d2516fbbfe47 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:10.849558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:10.850546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:10.851502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 393/789] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:10.859764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:10.894240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:10.895724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:10.896190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:10.896959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:10.912095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:10.919826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:10.921051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:10.922100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:10.924666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 397/790] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:10.930747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.010276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a2c4c-0c15-48a0-9b84-a5ad59223c71 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.020563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.071010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.071765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.072048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.072321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.073062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:11.073361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.075659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.104044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 394/791] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:11.109559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.118155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a05af8-71cd-4a29-b0e2-2359b5a952f2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.168256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.169780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.170264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.170768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.171653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:11.172695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.174582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6088-8a9a-464f-888b-9cee1be16d9e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.187481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 398/792] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:11.189294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6539d14-0ad5-4b2f-90d2-9553f9835d33 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.190042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6539d14-0ad5-4b2f-90d2-9553f9835d33 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.190562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6539d14-0ad5-4b2f-90d2-9553f9835d33 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.193301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 399/793] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:11.194217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6539d14-0ad5-4b2f-90d2-9553f9835d33 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.210044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.231144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b480b8-8c3e-4660-b108-84c69585fd4a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.273888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.275140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.275631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.276118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.289526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.292505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6207e7e5-2c65-417e-a0c5-6b29559df2df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:11.310372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:11.311378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:11.312266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:11.312266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:11.312266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:11.314586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 395/794] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:11.316167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.316701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.317171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.337097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:11.338940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:11.352403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.423304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 396/795] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:11.424275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.451053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6539d14-0ad5-4b2f-90d2-9553f9835d33 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.495304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.495966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.496249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.496520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.508588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.511214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-6dac54e6-a823-4e47-afb1-a464dfb4a0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:11.528391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:11.529402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:11.530299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:11.530299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:11.530299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:11.532510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 400/796] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:11.533402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d41b12-2ae2-4b22-bc67-2b462c0f78cc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.534154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c9722-0f7e-4b62-aefc-88b9d197888a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.534677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c9722-0f7e-4b62-aefc-88b9d197888a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.535146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c9722-0f7e-4b62-aefc-88b9d197888a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.540956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 401/797] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:11.542375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c9722-0f7e-4b62-aefc-88b9d197888a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.577291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.577998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.578231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.578552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.579175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:11.579483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.580895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863cb58a-c76e-45a0-ab47-818a7ea36434 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.593564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 397/798] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:11.594818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.595103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.595375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.608679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.611281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.615461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:11.616698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:11.660053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c9722-0f7e-4b62-aefc-88b9d197888a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.702439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 398/799] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:11.702439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.705912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.706862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.707356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.707831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.708750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:11.709268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.710995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.737389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 402/800] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:11.738568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.818746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149be53-a12a-4164-8d2c-be9a4db490a9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.861507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e08148-ec20-4c7a-98c7-d25056848888 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:11.867507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.868256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.868764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.869641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.873529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.881180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.883650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.890974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:11.904699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:11.905764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:11.906807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:11.908985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:11.909865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:11.910373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:11.910846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:11.911405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 399/801] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:11.912265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:11.913421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:11.914083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:11.929953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:11.931087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:11.937229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 403/802] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:11.938035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.042946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092fdda-b615-4e19-9d76-581825593f82 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.055872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b4954-80f9-47c0-8214-2a36e7a4723d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.078516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.079251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.079539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.079816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.086834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.088908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.093170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.093770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.094046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.094313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.095020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:12.095320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.096751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:12.100537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:12.101456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:12.102431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:12.106275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 400/803] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:12.107251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.108862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 404/804] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:12.114234 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.161735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:12.165993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:12.210571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3a36c-5650-4aab-bc62-7e72cee21a6f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.221087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033b311-0312-44b8-b03a-2d3301e3933e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.248329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.249011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.249311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.249670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.250300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:12.250630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.252215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.254607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.255230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.255504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.255770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.262113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 401/805] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Sep 3 01:15:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:12.262113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.271836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:12.277586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:12.278562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 405/806] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:15:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:12.279435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.371208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50fb2d-ab54-4915-89ae-1cc0a7d09fa9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.386012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aea797-b666-42f0-844b-9e0609aac590 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.388752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:12.392742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:12.408249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.408862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.409173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.409456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.410180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:12.410531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.411977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.422424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.423210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.423490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.423762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/0711b98a2c214b7cb807f2d211b628c7/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.425683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=0711b98a2c214b7cb807f2d211b628c7, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.427195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.430489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 402/807] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:15:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:12.431597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.533891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43705b32-ed7a-494d-a369-580a89e2b810 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.548089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 406/808] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:15:12 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/0711b98a2c214b7cb807f2d211b628c7/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:15:12.549255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.589551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.590406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.590766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.591131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.592150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:12.592523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.594001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.605702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 403/809] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Sep 3 01:15:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:12.605702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.660700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:12.716402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c1b27-4a30-48f8-9740-3cc12fc5c189 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.735991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c5b35-9a59-4830-a766-93cc79b9aa33 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.762689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.763304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.763583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.763853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.764522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:12.764823 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.766198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.775637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 407/810] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:15:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:12.776469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:12.784753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e0ea8-3c0b-441c-8a76-42e1231a7eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.785379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e0ea8-3c0b-441c-8a76-42e1231a7eca None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.785659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e0ea8-3c0b-441c-8a76-42e1231a7eca None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.785938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e0ea8-3c0b-441c-8a76-42e1231a7eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.810896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:12.896983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450f3ff-a5de-4801-a3be-f49178a95b4c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:12.933746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:12.934304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:12.934608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:12.934877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:12.935543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.936909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:12.986857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e0ea8-3c0b-441c-8a76-42e1231a7eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:12.988303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e0ea8-3c0b-441c-8a76-42e1231a7eca None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.030380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:13.068581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 408/811] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:13.069714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.098453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 404/812] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:15:13.100572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282ef6-d258-4356-846b-d668fa6a4a35 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.101033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282ef6-d258-4356-846b-d668fa6a4a35 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.101578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f282ef6-d258-4356-846b-d668fa6a4a35 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.107447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 405/813] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:13.107447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f282ef6-d258-4356-846b-d668fa6a4a35 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.202979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:13.233972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f282ef6-d258-4356-846b-d668fa6a4a35 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.239760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43deabf0-a1b4-4df8-8303-27dcae7db421 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.280649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:13.281289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.281566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.281831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.283170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:13.283806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.284142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.284415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.289701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:13.296961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:13.297263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:13.298459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.299346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4d95b793-d386-45f2-be15-0307e164f2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:13.316028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:13.317017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:13.318030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:13.318030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:13.318030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:13.318200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:13.319072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:13.320013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:13.320013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:13.320013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:13.320181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 406/814] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:13.320181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.320303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 409/815] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:13.320379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.327028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.327324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.350109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:13.351651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:13.437205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362f016-016e-4ea2-a225-e45bb9bc5e21 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.438469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 407/816] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:13.439379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.470789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:13.479191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:13.479972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.480265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.480542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.481200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:13.481911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:13.483359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.497703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 410/817] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:13.504091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.639288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50879f-4fbc-4b0b-a6cf-7baa1fe3135a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.675174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:13.689660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45826-6b6a-4c38-9c2e-c117ce4a8c16 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.708291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:13.709140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.710311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.710907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.711931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:13.714211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:13.715798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.738024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 408/818] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:13.744151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.779909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:13.781144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.781619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.782031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.810361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:13.810361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:13.821210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:13.822218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:13.823277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:13.828072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 411/819] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:13.834383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.861514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e52683-ab97-439d-9d8a-2d18449384a0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.893110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:13.909739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:13.910562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:13.910842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:13.911112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:13.920384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:13.922362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:13.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:13.944765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 409/820] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:13.945888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:13.957403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b0dc7-a5d0-4a5c-8a67-0d34361f05c8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:13.993024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.005981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae418094-061d-4c15-a393-467050606742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.006951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae418094-061d-4c15-a393-467050606742 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.007902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae418094-061d-4c15-a393-467050606742 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.008642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae418094-061d-4c15-a393-467050606742 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.010125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae418094-061d-4c15-a393-467050606742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:14.010653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae418094-061d-4c15-a393-467050606742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.013004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae418094-061d-4c15-a393-467050606742 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.038350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 412/821] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:14.039494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae418094-061d-4c15-a393-467050606742 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.074098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7e6c6-9aa5-4e87-9176-81c76a9bfd49 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.111631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.112257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.112626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.112968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] PATH_INFO: `/v3/services/network` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.113610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.115166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:14.125257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:14.126254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 03 01:15:14.127129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:14.130906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 410/822] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:13 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:14.131793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.178126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae418094-061d-4c15-a393-467050606742 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.189912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.216022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.216773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.217183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.217560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.218630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:14.219000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.220943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.239473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 413/823] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:14.239473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.267149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d19f89-61d6-4dbf-b58b-d7a6c9b4c93a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.271711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.307599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.308451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.308827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.309191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.320065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.322389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:14.337700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:14.338860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:14.339972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:14.339972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:14.339972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:14.345889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 411/824] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:14.346881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.370703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330fc51-31d4-45a1-a09a-9fe45d92bdb1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.418751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.420354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.420631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.420896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.421520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] RBAC: Adding query filter params (name=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:14.421814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.423220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.434295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 414/825] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:14.434295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.469264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.472209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a269a1f-a87a-4268-8f44-2747b7a5d652 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.518769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.519303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.519583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.519909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.520950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=1749220d9e9547eba7ea035508c365bd) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:14.521255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.522777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.534366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 412/826] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/role_assignments?user.id=1749220d9e9547eba7ea035508c365bd&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:14.534366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.566641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c875ed-26e9-4581-9986-4f2d58956c21 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.575103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.633168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.633768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.634251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.634735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.635652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:14.635963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.637377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdee53-5218-491d-ba80-68a3d183a680 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.655351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 415/827] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:14.662996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa250ab-150d-4302-b0f3-7b04f9899f65 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.663512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa250ab-150d-4302-b0f3-7b04f9899f65 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.664275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa250ab-150d-4302-b0f3-7b04f9899f65 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.666913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 416/828] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:14.668072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa250ab-150d-4302-b0f3-7b04f9899f65 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.694871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280bc5c-1252-4607-9a6b-84bbb66a9c14 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.741632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.743806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.743806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.744157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.744757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] RBAC: Adding query filter params (type=network) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:14.746016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.746682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e2ba4-4a0f-4284-a572-13db7a994e13 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.754397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 413/829] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:14.755471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.755755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.756051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.780400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:14.781889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:14.813160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.828429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa250ab-150d-4302-b0f3-7b04f9899f65 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:14.864097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 414/830] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:14.865319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:14.875799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:14.876629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:14.877051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:14.877523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:14.897480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:14.897693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:14.901297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:14.923558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:14.924909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:14.925963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 417/831] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:14.935351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.025876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24851f4b-cd74-43dd-85f8-e811eee486b0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.100080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:15.100080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.100080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.100080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.100497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:15.100650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:15.145862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829c8c6-67b5-4291-8e7f-0666f212a0a8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.187367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:15.187977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.188255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.188523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.189214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:15.189518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:15.190980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c77ec-96bc-40b8-84b6-ed85414eac9e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:15.202527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 415/832] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:15.203054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.205406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 418/833] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:15.207131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70d55-0fa6-4ea1-88af-2ab323212582 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.207574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70d55-0fa6-4ea1-88af-2ab323212582 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.207992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e70d55-0fa6-4ea1-88af-2ab323212582 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.209864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 419/834] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:15.211443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e70d55-0fa6-4ea1-88af-2ab323212582 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.230661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:15.321807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:15.337613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e70d55-0fa6-4ea1-88af-2ab323212582 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.368442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cf25f-8a8b-4c6d-8f86-ac8d6f207c22 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.375961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:15.376729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.377074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.377416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/57efcdf1033744358788130e1c663538/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.393920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=57efcdf1033744358788130e1c663538, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:15.395653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:15.417142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f518b-c501-4eb0-afae-fb9729f5527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:15.418059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f518b-c501-4eb0-afae-fb9729f5527a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.418583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f518b-c501-4eb0-afae-fb9729f5527a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.419076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f518b-c501-4eb0-afae-fb9729f5527a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.429562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f518b-c501-4eb0-afae-fb9729f5527a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-051f518b-c501-4eb0-afae-fb9729f5527a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:15.435312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:15.437216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 416/835] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:15.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.439202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.439202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.457858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:15.459225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:15.463297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 420/836] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:15:15 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/57efcdf1033744358788130e1c663538/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:15:15.464143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.547142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:15.549841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 417/837] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:15.608348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89f5d-9a07-429d-9a85-f145bda9419d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.648275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:15.648814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.649093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.649357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.650764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:15.651142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:15.652520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:15.661654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 421/838] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:15.667140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.732848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:15.776053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe1dc6e-0d66-4d8b-858d-08a680287275 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.811894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:15.812558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:15.812839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:15.813120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:15.825772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:15.827305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:15.841383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:15.842854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:15.844988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:15.847117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:15.847117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:15.847117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:15.847117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 418/839] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:15.849958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:15.979501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fc685-f3a8-468c-aac7-d4f6ed393959 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:15.986940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f93a1c-398c-4d17-9d40-c0884a8bb642 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.004323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:16.009647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:16.030609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.031733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.032244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.032882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.034170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.035729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.036215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.036996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.037282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.041233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.041233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:16.041233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.041233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:16.046719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:16.047890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:16.048990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:16.051928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 422/840] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:16.062440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 419/841] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:16.063876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:16.065169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:16.190870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebedd3-2d2a-45c4-845e-c664cb2316f3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.195248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999daf7-7c6a-402d-9880-725d9d87914e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.231582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.232216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.232451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.232781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.240108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.241548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.242273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.243491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.243995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.245003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.245874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:16.246453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.248049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:16.253865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:16.255037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 03 01:15:16.255965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:16.257001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 423/842] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:16.262475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 420/843] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:16.265249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:16.265359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:16.299594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:16.335705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:16.384867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360efe3e-12a6-4fb3-badd-b2094ad3c79a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.392074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1370caf-6f4d-44fc-9a16-854c1597a96f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.429169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.430121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.430671 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.431167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.431401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.432024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.432311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.432587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.432813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:16.433312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.433423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:16.433622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.435095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8fa5d-ad2e-4aaf-8eff-36df2ff19d11 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.435432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.447067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 424/844] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:16.456502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 421/845] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Sep 3 01:15:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:16.462479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:16.582809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c25fd1-0951-4699-b7e4-3ff581d4ee51 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.598292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:16.621130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.621769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.622052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.622327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.695432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.697646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:16.706463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3d82-6444-4442-a150-481f5734d013 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.706760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3d82-6444-4442-a150-481f5734d013 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.707039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3d82-6444-4442-a150-481f5734d013 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.708631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 422/846] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:16.713732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.714016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.714291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.733020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:16.734210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:16.816949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 425/847] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:15:16.822854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:16.829013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 423/848] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:16.841770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:16.937366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598a3f-3766-47e9-8f4a-9d8ce32a600c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:16.975654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:16.976683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:16.977160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:16.977684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:16.988528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:16.990265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:17.007195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:17.008201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:17.009237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:17.009237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:17.009237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:17.011080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 426/849] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:17.016841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.040165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:17.107395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.126723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614698f-89d6-4a12-83d6-d4d31ece94d2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.163357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.163975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.164256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.164527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.165309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:17.165613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.167298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.183539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 424/850] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:17.183700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:17.184975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.234763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267bc60-72d9-43ce-b7fe-e01b293cbd84 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.282047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.283159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.283649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.284125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.285435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.287559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.290812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:17.302099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:17.304138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:17.305302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:17.312318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 427/851] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:17.313219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.319192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30c70e-cdd1-4b61-be44-24ae39cb7523 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.355577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.356210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.356484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.356760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.363796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.365191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:17.376952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:17.377916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:17.378843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:17.382914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 425/852] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:17.384455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.446109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af23d1-73b5-4228-8026-7733fe4900ee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.478681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:17.486312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.487022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.487303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.487575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.492446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:17.492446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.492446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e580032-df38-4621-b1b4-9be3b61e74c8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.495328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 428/853] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:17.501091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7bfc-25ae-4ec5-af74-7816d3036c3c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.501386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7bfc-25ae-4ec5-af74-7816d3036c3c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.501666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b7bfc-25ae-4ec5-af74-7816d3036c3c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.507071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 429/854] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:17.507071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7bfc-25ae-4ec5-af74-7816d3036c3c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.512007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be50d9-b89e-4830-880c-b9db6e65a434 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.547972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.548578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.548852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.549124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.549810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:17.550115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.551587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c92a64-0223-405a-b0e6-ae5ddbee946a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.560837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:17.565004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 426/855] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:17.566174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.566488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.566761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.591476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:17.592730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:17.619482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b7bfc-25ae-4ec5-af74-7816d3036c3c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.654655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.655485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.655914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.656192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.656822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:17.657122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.658531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.662893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 430/856] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:17.663717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:17.678382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 427/857] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:17.730765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:17.777885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eafe318-6e82-4175-b1c6-ebaa017e9a5e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:17.829105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:17.829723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:17.830004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:17.830276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:17.831044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=0711b98a2c214b7cb807f2d211b628c7) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:17.831354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:17.832792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:17.840458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 431/858] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/role_assignments?user.id=0711b98a2c214b7cb807f2d211b628c7&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:17.965991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.084920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b14bd-2f77-4719-8958-30ff8b7f4bf8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:18.116387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:18.122499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:18.123144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.123424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.123691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.134751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:18.139526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:18.140472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:18.141320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 428/859] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:18.148703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.255312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646339f4-80b6-40f9-a050-d14f3d9759ed None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:18.278122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:18.296091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:18.296708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.296993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.297267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.297948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:18.298250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:18.299669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c333c0e-8a5c-42cc-bb80-c138df27bf51 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:18.309962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 432/860] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:18.315311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.431339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be218586-17ba-4956-8aeb-cdcf77e0aa8d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:18.460776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6ad9a-176e-4e9a-afb7-f996c16bf39b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.461210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6ad9a-176e-4e9a-afb7-f996c16bf39b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.461623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6ad9a-176e-4e9a-afb7-f996c16bf39b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.467677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 433/861] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:18.473559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.474093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.474464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:18.474903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.475109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.475401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.475678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.487079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:18.488541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:18.495873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:18.497368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:18.504230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:18.505161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:18.506773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:18.506773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:18.506773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:18.506773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 429/862] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:18.512190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.567253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:18.580070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 434/863] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:18.630548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850dc2d-3425-4793-8ea1-b9108fdb8a5b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:18.666490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:18.667106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.667394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.667666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.668308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:18.668614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:18.669991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:18.686676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 430/864] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:18.688232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.795248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680eb1c-950d-4fa9-a58b-429d4d9aa01a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:18.809904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:18.833181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:18.833826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:18.834108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:18.835240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:18.842251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:18.843798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:18.852276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:18.857859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:18.858837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:18.860380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 435/865] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:15:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:18.864947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:18.962923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdde69-e850-4f32-aa32-958de5d80945 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:18.964519 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:18.992532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44542f3f-db66-42e0-9976-cb839a67a7e8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:19.000973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:19.001597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.001882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.002269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.002987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:19.004404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:19.032593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:19.033206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.033489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.033765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.034529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:19.034839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:19.036360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:19.054027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 436/866] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Sep 3 01:15:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:19.060127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:19.081136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 431/867] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:18 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:19.094307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:19.173700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7506f-097b-494a-8028-3d8a0487db1f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:19.218397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:19.219024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.219609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.219879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.293768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:19.295263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:19.322698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:19.398884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 437/868] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:19 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 01:15:19.405093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:19.512129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465988-1e25-4846-b255-6fb8c28fc91a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:19.548747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:19.549742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.550253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.550752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.562425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:19.564186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:19.579748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:19.580708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:19.581606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:19.581606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:19.581606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:19.583448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 432/869] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:19.589422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:19.695526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9ad28-a6ed-4115-9259-1463129f8796 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:19.721662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:19.730908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:19.731514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.731792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.732062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.732705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:19.733001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:19.734381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5152c2-847e-48c5-a8ad-429f110aad8c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:19.748050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 438/870] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:19.753438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:19.773527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:19.789422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9983ae-b1b8-4dba-9cae-c71c7a01e722 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.789763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9983ae-b1b8-4dba-9cae-c71c7a01e722 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.790617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9983ae-b1b8-4dba-9cae-c71c7a01e722 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.792609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 439/871] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:19.802838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.803323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.803753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.825991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:19.827583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:19.869691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b26314-1ccf-4490-980e-94e0a460ff85 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:19.910363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:19.910363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:19.910363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:19.910363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:19.911345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 440/872] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:19.916337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:19.917804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2552d5aa-ec20-4b93-8465-f5a3f2961ee2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:19.930233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:19.932160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:19.933076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 433/873] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:19.938331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:20.041207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2051-00fb-4222-8495-c92c488ff547 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.041511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2051-00fb-4222-8495-c92c488ff547 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.041786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b2051-00fb-4222-8495-c92c488ff547 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.043516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 434/874] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:20.044764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32775cd-4cbe-46e0-b31e-3548da160454 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:20.048732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.049051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.049286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.061805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:20.069565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:20.071433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:20.080481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:20.081019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.081302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.081561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.082250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:20.082595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:20.084015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:20.097911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 441/875] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:20.105235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:20.162753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 435/876] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:20.197316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:20.201874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:20.217573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436452a-57e9-4334-b914-1d018824a09c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:20.257730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:20.258382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.259285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.259584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.260535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=57efcdf1033744358788130e1c663538) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:20.260848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:20.262935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:20.271644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 442/877] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:20 2022] GET /identity/v3/role_assignments?user.id=57efcdf1033744358788130e1c663538&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:20.333594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a3dd1-4643-4751-8f32-97bbd330aa52 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:20.375262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:20.376272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.376789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.377290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.378210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:20.379968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:20.444323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:20.469175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 436/878] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:20 2022] POST /identity/v3/services => generated 269 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:20.493982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:20.564668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c541ca-96c3-40c0-b3ea-56bd301a5fdb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:20.610991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:20.611635 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.611914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.612183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.612763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:20.614196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:20.624295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:20.625571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:20.626451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 443/879] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:20.631657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:20.703475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:20.745228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6e91e-0d0d-4d97-8a5c-2e5d51d70bf2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:20.781622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:20.782664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.783128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.783577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.784389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:20.784889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:20.786479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe254-16a7-477b-887b-b9446a1df4db None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:20.792120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 437/880] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:20.798290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:20.906588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:20.909601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f7b15-28ad-4db2-b3ee-7a87bd905ca8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:20.945015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:20.945782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:20.945978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:20.947126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:20.947764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:20.948068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:20.949447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e524987-8114-4fda-b752-de6a5f0797e3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:20.955042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 444/881] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:21.117616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f114c5-b03e-4fcf-9bfb-ac4b1061a981 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:21.117917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f114c5-b03e-4fcf-9bfb-ac4b1061a981 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:21.118261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f114c5-b03e-4fcf-9bfb-ac4b1061a981 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:21.119906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 438/882] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:21.125110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:21.125395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:21.125669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:21.145214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:21.146679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:21.224202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 445/883] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:21.500305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f114c5-b03e-4fcf-9bfb-ac4b1061a981 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:21.611677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f114c5-b03e-4fcf-9bfb-ac4b1061a981 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:21.639083 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:21.646952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:21.647585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:21.647868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:21.648137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:21.648710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:21.650152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:21.659186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:21.663177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:21.664022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 439/884] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:21.666730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:21.767969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:21.775250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b000d-a59a-4ca0-9987-5da4b5b3adce None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:21.809415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:21.810029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:21.810311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:21.810607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:21.811238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:21.811540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:21.813125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:21.818116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 446/885] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:21.829321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:21.931858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064d0ba-c661-42af-9582-d2cc439710f0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:21.967931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:21.968555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:21.968833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:21.969172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:21.969980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:21.970298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:21.971716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:21.978046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 440/886] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:21.987455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:22.101976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00b8ab-7c2b-48b8-9c3c-ba8802ac4918 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:22.139264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:22.139878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.140156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.140427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.141123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] RBAC: Adding query filter params (interface=public, service_id=67f92a1add4a4296a5c8c654f376df97, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:22.141430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:22.142825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da7ea-d415-489e-852d-e9ab6518574d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:22.148691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 447/887] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:21 2022] GET /identity/v3/endpoints?service_id=67f92a1add4a4296a5c8c654f376df97&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:22.154731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:22.217054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:22.260980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b8cda-a6b5-48c7-b0b1-9585c12d0b67 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:22.300609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e130c7-3297-4251-81ae-25872299ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:22.301587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e130c7-3297-4251-81ae-25872299ec14 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.302068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e130c7-3297-4251-81ae-25872299ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.302585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e130c7-3297-4251-81ae-25872299ec14 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.303421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e130c7-3297-4251-81ae-25872299ec14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:22.304991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e130c7-3297-4251-81ae-25872299ec14 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:22.311529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 441/888] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:22 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:15:22.448564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9328a-a2cc-4f2a-bcff-cee6bdc1e839 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.449049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9328a-a2cc-4f2a-bcff-cee6bdc1e839 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.449049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9328a-a2cc-4f2a-bcff-cee6bdc1e839 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.450725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 448/889] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:22.455905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40bfa-1401-49e1-9089-5a363a2e992c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.456196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40bfa-1401-49e1-9089-5a363a2e992c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.456467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e40bfa-1401-49e1-9089-5a363a2e992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.477732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c1e40bfa-1401-49e1-9089-5a363a2e992c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:22.479120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c1e40bfa-1401-49e1-9089-5a363a2e992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:22.552699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 442/890] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:22.582462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fea05-73bc-40e8-8e22-e57469581509 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.583068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fea05-73bc-40e8-8e22-e57469581509 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.583533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fea05-73bc-40e8-8e22-e57469581509 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.585076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 449/891] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:22.591825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.592350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.592833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.611817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:22.613348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:22.671899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:22.685889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 443/892] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:22.828152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fea05-73bc-40e8-8e22-e57469581509 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:22.854546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:22.936404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fea05-73bc-40e8-8e22-e57469581509 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:22.971764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:22.972399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:22.972677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:22.972948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:22.984348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:22.989338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:22.994281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:22.995164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 450/893] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Sep 3 01:15:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:23.005936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.006116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.093198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.111416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32afe09-8ef6-4b7b-82d0-65b08684cbd0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.114869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc72080-9ddd-41fc-93c8-96c4b4f80e1d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.147465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.148100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.148377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.148688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.149767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.150733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.151176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.151686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.152113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.152526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.153327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:23.153784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.155383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550b780-00d7-4394-9b79-163a1d449001 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-21ec0639-0aa8-44c0-b77d-61d551422c3d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:23.159959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:23.162426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:23.163395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 451/894] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:23.164190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5186f3-8438-4d5b-9a4a-9b9addc8b5ce None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.164190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5186f3-8438-4d5b-9a4a-9b9addc8b5ce None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.164190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5186f3-8438-4d5b-9a4a-9b9addc8b5ce None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.169536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 452/895] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:23.169536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5186f3-8438-4d5b-9a4a-9b9addc8b5ce None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.176241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 444/896] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:23.178243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.178919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.179209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.197749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:23.198977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:23.237249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.274619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 445/897] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:23.274619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.290960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5186f3-8438-4d5b-9a4a-9b9addc8b5ce None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.341415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.342563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.343459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.343888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.344924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:23.345628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.347895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.356315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 453/898] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:23.360729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.377317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.390802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffb0d7-484b-4270-b132-52ecbf6129c8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.429399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.430683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.431169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.431763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.443333 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.445100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:23.463404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:23.464364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:23.465239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:23.465239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:23.465239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:23.467515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 446/899] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:23.472798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.482882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422438d-d532-4f34-900a-1cbd5340af63 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.504367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.526388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.527008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.527287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.527559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.528208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:23.528858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.530613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.541813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 454/900] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:23.542834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.594161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03522d-a7e9-49aa-9356-42a9a5b5fe1a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.628512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.628684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.629256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.629655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.629786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.630460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:23.630760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.632520 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.644442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 447/901] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:23.645229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.661819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ff386-425e-40fb-8337-cf724978d8f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.702584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.703441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.703558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.703742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.704445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] RBAC: Adding query filter params (interface=public, service_id=450f750692ee420faac4b084278869da, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:23.704753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.706142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.715856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 455/902] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/endpoints?service_id=450f750692ee420faac4b084278869da&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:23.715856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.741505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.758149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67344669-dbf5-4598-88f7-39f3003e4d59 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.796001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.796608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.796899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.797175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.798382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.799253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.834613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74ba4-6709-4525-8640-42154bb01a86 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:23.870497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 448/903] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:23 2022] POST /identity/v3/services => generated 241 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:23.870497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.888900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:23.889820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:23.890369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:23.890831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:23.893470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:23.901995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:23.904561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:23.919199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:23.920466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:23.921669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:23.924696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 456/904] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:23.934390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:23.997492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9977a-5516-459b-8cd0-0364e68de121 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.041551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:24.042185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.042850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.043136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.043761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:24.045335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:24.053269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 449/905] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/services => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:15:24.069043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06a527-b4ce-456b-8802-9352cf08f460 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.115228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:24.116262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.116740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.117206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.119095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:24.119724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:24.122282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b27eb-42e7-4a5e-9c77-27b5c319b413 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:24.148555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 457/906] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Sep 3 01:15:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:24.154665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:24.168046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:24.267176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba7aa-ff68-4a8e-9fed-635288b6e463 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.286045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897fc6-c987-402e-9785-c343e688028e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.287215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897fc6-c987-402e-9785-c343e688028e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.287497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a897fc6-c987-402e-9785-c343e688028e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.288964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 458/907] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:24.294999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.295633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.296226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.307798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:24.308636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.309064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.309516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.315980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:24.317278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:24.370365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:24.398273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:24.400198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 459/908] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:24.400700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:24.411076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:24.504611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 450/909] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:15:24.510805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:24.616115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42589ac7-ddc7-49cf-8485-978b205ed92f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.652193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:24.653091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.653575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.654055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.664802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:24.666529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:24.681520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:24.682519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:24.683428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:24.683428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:24.683428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:24.688643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:24.689275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 460/910] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:24.691699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:24.738769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:24.788708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a205e-2a78-49b1-8be0-b6736f4e4175 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.802969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:24.808176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64587812-53cf-49e0-854d-159f8fa24880 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.828633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:24.829617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.830046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.830609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.831719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:24.833377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:24.847357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:24.848583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 03 01:15:24.849686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:24.852324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:24.853197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 451/911] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:24.853357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:24.858114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:24.858434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:24.859107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:24.859420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:24.859898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:24.860790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:24.879385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 461/912] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:24.880248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:24.984093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa42eab-4c42-44e6-9843-82b97d9ed193 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:24.986821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:24.998051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632ab05-9983-4a52-ad22-4a6b211b4b5a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:25.030424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:25.031476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.031859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.032210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.038152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:25.039119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.039619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.040132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.040991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:25.041532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:25.041966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:25.043452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:25.043565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:25.049772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 452/913] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:25.058446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:25.058592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:25.059504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:25.060422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 462/914] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:25.067878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:25.105257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:25.186818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d633-4bd0-4705-822c-ee64a2933095 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:25.198549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8b41d-a94a-4271-a1f7-96d3d542ee5a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:25.235995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:25.237219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.238098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.238717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.239376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:25.239686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:25.241078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:25.247270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:25.248117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.248295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 453/915] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Sep 3 01:15:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:25.248727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.249074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.249773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:25.250076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:25.258630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:25.258785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:25.266375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 463/916] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:25.272223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:25.290886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:25.360500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:25.363341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f0fe3-ac89-4fae-873c-58e2117e604e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:25.385827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce7b0-dda4-473c-9428-72c3fbdf8979 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:25.398181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6ee30-7b50-460a-a0d1-fad3ff6bd8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:25.399699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6ee30-7b50-460a-a0d1-fad3ff6bd8f4 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.400028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6ee30-7b50-460a-a0d1-fad3ff6bd8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.400698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6ee30-7b50-460a-a0d1-fad3ff6bd8f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.402010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6ee30-7b50-460a-a0d1-fad3ff6bd8f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:25.403537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c6ee30-7b50-460a-a0d1-fad3ff6bd8f4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:25.426067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:25.426713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.426995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.427377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.432689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] RBAC: Adding query filter params (role.id=f4b01b45806047c39ce1eefde1394038, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=57efcdf1033744358788130e1c663538) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:25.432689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:25.432689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ddc2dd-2ad1-4f51-b42d-8d66aaf36167 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:25.436107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 464/917] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:25 2022] GET /identity/v3/role_assignments?user.id=57efcdf1033744358788130e1c663538&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f4b01b45806047c39ce1eefde1394038 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:25.474573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 454/918] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:25.889041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fed8be-b517-4759-b173-145112501c17 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.889542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fed8be-b517-4759-b173-145112501c17 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.889639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fed8be-b517-4759-b173-145112501c17 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.891188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 465/919] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:25.896886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00586f90-8388-47de-afa2-3625ce897c3c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:25.897307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00586f90-8388-47de-afa2-3625ce897c3c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:25.897480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00586f90-8388-47de-afa2-3625ce897c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:25.917103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-00586f90-8388-47de-afa2-3625ce897c3c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:25.918313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-00586f90-8388-47de-afa2-3625ce897c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:25.999415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 455/920] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:26.100573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e716e1-02ca-4bc5-a4b4-c50fb4210f44 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:26.100873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e716e1-02ca-4bc5-a4b4-c50fb4210f44 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:26.101092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e716e1-02ca-4bc5-a4b4-c50fb4210f44 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:26.103474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 466/921] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:26.108793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:26.109110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:26.109393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:26.128104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:26.129298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:26.205871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 456/922] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:26.291238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e716e1-02ca-4bc5-a4b4-c50fb4210f44 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:26.397569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:26.429558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e716e1-02ca-4bc5-a4b4-c50fb4210f44 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:26.479348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:26.480341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:26.480849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:26.481317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:26.482107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:26.483696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:26.496035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:26.497052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:26.497906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:26.501880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 467/923] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:26.508246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:26.525830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:26.625019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66d24e-1078-4db0-934f-de62ac7cf39f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:26.668493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:26.676536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:26.677173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:26.677457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:26.677741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:26.678408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:26.678770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:26.681259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:26.692425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 457/924] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:26.700698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:26.844604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:26.878874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a1b32-c2b2-4e16-9a47-f4001aa98d64 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:26.898573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52751cca-ccb2-4a36-b0cd-725917cd0a31 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:26.927795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:26.928515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:26.928890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:26.929290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:26.930259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:26.933436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:26.945582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:26.946644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 03 01:15:26.947632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:26.950468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 468/925] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Sep 3 01:15:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:26.955951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:26.959465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:26.960088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:26.960372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:26.960645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:26.961288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:26.961620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:26.963140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:26.964567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:26.974618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 458/926] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:26.976265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:27.066130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eeb2e4-f8fd-4704-8437-cd7bcf433db8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:27.089064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490561ba-0832-473a-a1a8-4167b2c05c27 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:27.102649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:27.103304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.103586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.103855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.104492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:27.104800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:27.106163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:27.115783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 469/927] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:27.116841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:27.138263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:27.138919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.139201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.139473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.140491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] RBAC: Adding query filter params (interface=public, service_id=41a0a1c350be4448a4e1d5d93e2039de, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:27.140800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:27.142383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:27.154259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 459/928] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:26 2022] GET /identity/v3/endpoints?service_id=41a0a1c350be4448a4e1d5d93e2039de&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:27.155440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:27.159882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:27.204901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:27.237087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf3a21-e5e2-422b-aae8-63d2b85b8233 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:27.271809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787fa4c0-399a-4f2c-9b7c-fd1e30d06d2b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:27.273626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:27.274475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.274761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.275041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.275670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:27.275972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:27.277349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:27.287253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 470/929] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:27.289119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:27.321182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a5058-01b5-407b-844b-ad87983ec639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:27.322365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5058-01b5-407b-844b-ad87983ec639 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.322657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5058-01b5-407b-844b-ad87983ec639 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.322930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a5058-01b5-407b-844b-ad87983ec639 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.323537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a5058-01b5-407b-844b-ad87983ec639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:27.325397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a5058-01b5-407b-844b-ad87983ec639 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:27.342374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 460/930] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:27 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:15:27.407814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffe946-6762-4aca-b21d-dc46dda8367a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.408236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffe946-6762-4aca-b21d-dc46dda8367a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.408365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffe946-6762-4aca-b21d-dc46dda8367a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.409992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 461/931] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:27.412979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a713f-d205-4a91-b2f9-1680ecf07436 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:27.415634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d2d6c-c360-440d-9cfc-90c524c00b1c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.415925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d2d6c-c360-440d-9cfc-90c524c00b1c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.416200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d2d6c-c360-440d-9cfc-90c524c00b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.435707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-be2d2d6c-c360-440d-9cfc-90c524c00b1c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:27.436920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-be2d2d6c-c360-440d-9cfc-90c524c00b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:27.453115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbe421-ea24-4705-9055-40c40316699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:27.453683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbe421-ea24-4705-9055-40c40316699f None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbe421-ea24-4705-9055-40c40316699f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.454292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbe421-ea24-4705-9055-40c40316699f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.454994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbe421-ea24-4705-9055-40c40316699f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:27.456645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbe421-ea24-4705-9055-40c40316699f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:27.463663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:27.512830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:27.518248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 462/932] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:27.558807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 471/933] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:27.636521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967517-5a08-483d-bb5e-6bf6421014db None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.636860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967517-5a08-483d-bb5e-6bf6421014db None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.637074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec967517-5a08-483d-bb5e-6bf6421014db None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.638812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 463/934] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:27.646389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.646389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.646389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.667281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:27.668769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:27.756042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 472/935] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:27.827253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec967517-5a08-483d-bb5e-6bf6421014db None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:27.936675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec967517-5a08-483d-bb5e-6bf6421014db None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:27.979661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:27.980308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:27.980589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:27.980858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:27.991433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:27.996344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:27.997525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:27.998386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 464/936] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Sep 3 01:15:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:28.005574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.030864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.116925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:28.119975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8ce69-288c-46bb-a0ab-e6e52a36637e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.146112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:28.157058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.157793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.158176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.158568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.159335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:28.159738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.161199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.170108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 473/937] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:28.174906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a584a-52fc-433f-8db8-20336c9ba38a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.179305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.210588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.211201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.212067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.212495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.213162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.214705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.217313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING keystone.server.flask.common [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 03 01:15:28.261944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.sql.core [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 01:15:28.261944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 01:15:28.261944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [parameters: {'id': 'bc82531526a24193a8e53499184c3d0f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 01:15:28.261944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81627) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc82531526a24193a8e53499184c3d0f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc82531526a24193a8e53499184c3d0f. Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 01:15:28.271179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 03 01:15:28.272003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application yield resource Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise self.value Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application session.commit() Sep 03 01:15:28.272960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.flush() Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application compat.raise_( Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 01:15:28.274009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 01:15:28.275075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 01:15:28.276991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [parameters: {'id': 'bc82531526a24193a8e53499184c3d0f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.278800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:28.280670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc82531526a24193a8e53499184c3d0f. Sep 03 01:15:28.282887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.285737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 465/938] 10.222.0.13 () {60 vars in 1140 bytes} [Sat Sep 3 01:15:28 2022] POST /identity/v3/projects => generated 262 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:28.291150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.303713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df197d8c-e1c2-4bd7-ad05-c417739d7812 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.349644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.350283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.350653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.350933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.365388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.367046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:28.387089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:28.388074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:28.388983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:28.388983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:28.388983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:28.394117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 474/939] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:28.394986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.406488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa00d-2939-41a7-b641-e38715a04c82 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.422578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:28.433504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:28.447058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.448058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.448559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.450442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.457567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.459769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:28.472659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:28.473591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:28.474530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:28.477966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 466/940] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:28.484244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.523026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aad7a9-7aa4-43e1-9b8e-c7ed120194d2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.560384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.560929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.561213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.561536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.562132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:28.562507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.563894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.580912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 475/941] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:28.581821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.610874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a539c9-aa23-4d11-847c-05aeba0f54e6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.648311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.649770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.650730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.651014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.651701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:28.652013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.653639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.686073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:28.686982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 467/942] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:28.698426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0db89-7a78-48ee-8375-1626b0d42d3f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.739098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.740016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.740448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.740900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.750707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.752508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:28.767988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:28.769373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:28.770923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:28.773327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 476/943] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:28.782546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:28.823158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:28.901323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c60dac-bbff-46e2-87fe-4682a60608d5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:28.939909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:28.940540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:28.940795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:28.941067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:28.941806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:28.942104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:28.943819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:28.964355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 468/944] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:28.969636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:29.075362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f69e0-4fe5-427f-ad34-42a7d9a11786 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:29.111294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:29.111929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.112212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.112485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.123430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:29.124905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:29.139782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:29.140812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:29.142105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:29.142105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:29.142105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:29.142105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 477/945] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:29.147175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:29.254492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:29.264201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455910-29e4-43c7-8edb-1ba027c0ca9a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:29.299630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:29.300247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.300874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.301404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.302584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:29.302956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:29.304399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba73ad-0c16-4438-9bcf-42ec3a4d58ae None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:29.316704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 469/946] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:29.323645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:29.333019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:29.390973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe5d5-7b45-489a-b925-99fc2b4b491a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.391264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe5d5-7b45-489a-b925-99fc2b4b491a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.391545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fe5d5-7b45-489a-b925-99fc2b4b491a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.394102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 470/947] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:29.398324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.398651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.398990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.417953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:29.419564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:29.441204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f413c9-9040-42ba-ae69-284e59a96d08 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:29.484934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:29.485980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.486626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.486967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.494610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:29.496156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:29.498463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 471/948] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:29.508059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:29.510085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:29.511826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 478/949] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:29.518667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:29.605225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:29.625096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea073af8-d537-4a44-8040-e5f61bd27c97 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:29.660260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:29.660870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.661152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.661423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.662115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:29.662444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:29.663878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295b349-98bf-45c6-9451-4a4c1ff7c9ad None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:29.675876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 472/950] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:29.682540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:29.683797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15092136-0e72-4228-873a-87fb527bc39d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.684087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15092136-0e72-4228-873a-87fb527bc39d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.684351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15092136-0e72-4228-873a-87fb527bc39d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.685766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 473/951] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:29.690745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.691031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.691304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.711414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:29.712993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:29.756222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:29.798696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 474/952] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:29.799998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98d88f-90ae-40c9-9e77-49234661b4e5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:29.807667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:29.849196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:29.849874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.850154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.850481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/57efcdf1033744358788130e1c663538/roles/f4b01b45806047c39ce1eefde1394038` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.852788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=57efcdf1033744358788130e1c663538, role_id=f4b01b45806047c39ce1eefde1394038)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:29.854232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:29.894790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:29.916410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 479/953] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:15:29 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/57efcdf1033744358788130e1c663538/roles/f4b01b45806047c39ce1eefde1394038 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:15:29.926231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fea5d2-9b3b-48d0-8a63-25ab1d2e6d15 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:29.972313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:29.973135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:29.973359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:29.973633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:29.974508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:29.975801 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:29.987317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:29.988556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:29.990068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 475/954] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:29.995496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:30.069659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:30.124153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f26c14-fde6-486a-9a11-3239bf5a6451 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:30.153361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:30.168521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:30.169136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.169412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.169678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.170312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:30.170640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:30.172018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:30.178644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 480/955] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:30.183988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:30.210124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79c8cc-e19e-4952-8d17-968a579f1608 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:30.247588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:30.248256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.248534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.248804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.249379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:30.251287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:30.254385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:30.263368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:30.264572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:30.266091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 476/956] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:30.271094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:30.310432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db8511-10b0-4c01-be6a-40a2d09b293a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:30.353547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:30.354246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.354561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.354828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.355441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:30.355739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:30.357099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:30.366590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 481/957] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:30.368772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:30.408912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab627ac-9f9a-4526-8496-ba71def3a06f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:30.433757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:30.454322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:30.455564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.456036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.456735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.458359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:30.458865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:30.460635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:30.469086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 477/958] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:30.473862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:30.500630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f1baf-5162-4169-a8cb-19d8f75da9df None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:30.539306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:30.548159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e59217-7917-47f9-bfa2-51bc1b465269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:30.548159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59217-7917-47f9-bfa2-51bc1b465269 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.548482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59217-7917-47f9-bfa2-51bc1b465269 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.552709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e59217-7917-47f9-bfa2-51bc1b465269 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.552709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e59217-7917-47f9-bfa2-51bc1b465269 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:30.552709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e59217-7917-47f9-bfa2-51bc1b465269 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:30.592205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93155158-3147-4cea-b56b-1a262d5824c5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:30.631088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 482/959] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:30.652150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:30.653210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.653727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.654235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.655215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:30.655733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:30.657490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f3b3d-634e-4561-9134-3ac62485a326 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:30.663881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 478/960] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:30.849984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c905-882d-4ddf-83a2-db0fa68afad7 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.850323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c905-882d-4ddf-83a2-db0fa68afad7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.850563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c905-882d-4ddf-83a2-db0fa68afad7 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.852799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 483/961] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:30.859133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:30.859451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:30.859692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:30.879680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:30.880880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:30.964812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 479/962] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:31.225581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c905-882d-4ddf-83a2-db0fa68afad7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:31.334897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c905-882d-4ddf-83a2-db0fa68afad7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:31.343160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:31.371807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:31.372427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:31.372711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:31.372984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:31.373643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:31.375192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:31.377801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING keystone.server.flask.common [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 03 01:15:31.415939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.sql.core [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 01:15:31.415939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 03 01:15:31.415939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [parameters: {'user_id': '0109c9d32f604be19fdb2cfde60d1623', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 03 01:15:31.415939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81628) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application conn.query(q) Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result.read() Sep 03 01:15:31.433110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 01:15:31.434052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application next(self.gen) Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application yield resource Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application next(self.gen) Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.session.rollback() Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.force_reraise() Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise self.value Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application session.commit() Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 03 01:15:31.435092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._prepare_impl() Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.session.flush() Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._flush(objects) Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application compat.raise_( Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application flush_context.execute() Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rec.execute(self) Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 03 01:15:31.436004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 03 01:15:31.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = self._query(query) Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application conn.query(q) Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result.read() Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 03 01:15:31.437782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [parameters: {'user_id': '0109c9d32f604be19fdb2cfde60d1623', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:31.440223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 03 01:15:31.442128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:31.448826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 484/963] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Sep 3 01:15:31 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:31.449846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:31.497102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:31.541754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bc9b7-11f5-46da-94ee-3584e18207d1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:31.577161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:31.577792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:31.578071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:31.578359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:31.585878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:31.587448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0c16e06f-39dc-4792-beb4-459a5426a659 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:31.599084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:31.600045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:31.601005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 480/964] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:31.607247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:31.713601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ae8ef-62ae-41aa-b1e1-80f4319507ca None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:31.749344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:31.750332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:31.750844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:31.751335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:31.752249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:31.752770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:31.754651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59f62-683e-482a-a26f-bd4857b3bafc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:31.797600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 485/965] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Sep 3 01:15:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:31.919388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a98c0c-ac9f-44a3-9c2f-06e81abc4cbd None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:31.919749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a98c0c-ac9f-44a3-9c2f-06e81abc4cbd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:31.920045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a98c0c-ac9f-44a3-9c2f-06e81abc4cbd None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:31.921759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 481/966] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:31.927829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:31.928147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:31.928390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:31.949576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:31.951136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:31.972890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:32.024048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 486/967] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:32.309944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a98c0c-ac9f-44a3-9c2f-06e81abc4cbd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:32.415798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a98c0c-ac9f-44a3-9c2f-06e81abc4cbd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:32.429574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:32.450941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:32.451663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.451855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.452128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.464235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:32.469052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:32.470001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:32.471198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:32.472283 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 482/968] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Sep 3 01:15:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:32.476356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:32.579839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26b56f-bd07-4565-b3fb-4822cf71699d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:32.614155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:32.614789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.615066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.615332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.615994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:32.616294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:32.618314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90269c21-7a98-41d1-953c-3975169fb5fc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:32.630826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 487/969] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:15:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:32.632700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:32.681381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476fc73-f87a-49ac-bcba-118de90c85ad None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.681674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476fc73-f87a-49ac-bcba-118de90c85ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.681956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476fc73-f87a-49ac-bcba-118de90c85ad None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.683477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 488/970] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:32.688163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dda41-fd0b-4633-80fc-4576b516d9d4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.688453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dda41-fd0b-4633-80fc-4576b516d9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.688731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dda41-fd0b-4633-80fc-4576b516d9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.707184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-389dda41-fd0b-4633-80fc-4576b516d9d4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:32.708383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-389dda41-fd0b-4633-80fc-4576b516d9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:32.723965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:32.745612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533150c-65f1-44eb-9fe7-c8892775e46d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:32.779271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:32.779906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.780191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.780464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.780555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 489/971] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:32.781801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aefe8-a6c1-4a95-8b35-e9a33c180e29 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.782090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aefe8-a6c1-4a95-8b35-e9a33c180e29 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.782583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873aefe8-a6c1-4a95-8b35-e9a33c180e29 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.783808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 490/972] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:32.788909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.789193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.789766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.795277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:32.796856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:32.811049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:32.812248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:32.812537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:32.813448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:32.819418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:32.819418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:32.819418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:32.819418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 483/973] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:32.819666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:32.861573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:32.884553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 491/974] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:32.940163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5db34-73a1-42db-b529-330ee52661d8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:32.973932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:32.974571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:32.974857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:32.975130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:32.975773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:32.976077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:32.978292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.039875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 484/975] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:33.041289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.050092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.123302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.143931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.145228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e0cc3-ecdd-4fff-818a-86bbedc10abf None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.160284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af57ca-2eeb-4062-ba05-f61af75b520b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.180753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.181397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.181680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.181953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.189023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.190486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.193964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.194616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.194899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.195170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.196602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.197972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:33.203471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:33.204606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:33.205567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 492/976] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:33.206200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:33.206781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:33.207709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:33.208545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 485/977] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:33.210154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.308499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe81e83-728c-4306-bb87-0140e1825717 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.309485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38c1ff-534c-4d63-b528-6ed576f04088 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.341939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.342574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.342854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.343123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.343405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.343764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.344007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.344285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.344553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.345131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.345341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:33.345645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.347144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.364505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 486/978] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:33.365301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.375412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.416676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.423370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 493/979] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:33 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:33.424237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.481116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ebd8e-2ef8-4224-a353-695dfb3e01f7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.530536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.531165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.531445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.531720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.532391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:33.532694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.534105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.542454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 487/980] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:33.545321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30600-562a-4af5-a972-25e5d1848572 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.546321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.587172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.588130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.588407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.588782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.620050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.654005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b447471-509f-4542-9e53-0c9c4b58a479 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.667237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.670707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.672234 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.690611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.691225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.691506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.691820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.692462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:33.692766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.694146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.704147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 488/981] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:33.705006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.784653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 494/982] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:15:33.785727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.815220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b8600-358c-45a3-a20a-4efcb7af0a3f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.848884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.855543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.856580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.858090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.858090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.858956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] RBAC: Adding query filter params (interface=admin, service_id=41a0a1c350be4448a4e1d5d93e2039de, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:33.859522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.860933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.872104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 489/983] 10.222.0.13 () {56 vars in 1249 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/endpoints?service_id=41a0a1c350be4448a4e1d5d93e2039de&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:33.873267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:33.906638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:33.911178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd2aed-26ec-4fab-8d00-28053a1ccd10 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.958381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:33.959303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:33.959589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:33.959860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:33.971483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:33.972928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:33.982684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084df004-53a1-4908-80b5-63de2be1ec45 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:33.987460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:33.988424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:33.994535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:33.994535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:33.994535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:33.994535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 495/984] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:33.994535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:34.022601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:34.023215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:34.023497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:34.023819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:34.024447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:34.025806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:34.042375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 490/985] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:15:34.120429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127c0ec-fce3-4e49-86bc-f2c14c0c3de1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:34.157805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:34.158367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:34.158644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:34.158909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:34.159559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:34.159855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:34.161229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:34.181779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 496/986] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:34.183317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:34.246479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:34.287950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a1ff-6cf8-4cb1-89c3-985011b26007 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:34.321897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:34.322552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:34.322832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:34.323099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:34.329700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:34.331173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:34.342815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:34.343709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:34.344625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 491/987] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:34.350998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:34.364209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:34.454170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2c203-9525-4558-99a0-bc56f478d9d9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:34.488749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:34.489364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:34.489644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:34.489915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:34.490636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:34.490942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:34.492514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dca3f-bd6b-42d5-8969-86564ee12e16 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:34.504674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 497/988] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:34.511370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:34.619119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d71076-14fa-4552-ad11-0d3e5da74309 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:34.639481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:34.654308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:34.654947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:34.655231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:34.655508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:34.656239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] RBAC: Adding query filter params (role.id=f4b01b45806047c39ce1eefde1394038, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=57efcdf1033744358788130e1c663538) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:34.656549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:34.658051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2052d6-eaa9-4e47-8a47-d19c523a8372 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:34.667426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 492/989] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:34 2022] GET /identity/v3/role_assignments?user.id=57efcdf1033744358788130e1c663538&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f4b01b45806047c39ce1eefde1394038 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:35.373571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbf730-f5ee-415f-9310-1b86216528c0 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:35.374115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbf730-f5ee-415f-9310-1b86216528c0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:35.374115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbf730-f5ee-415f-9310-1b86216528c0 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:35.375653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 498/990] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:35.380768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:35.381061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:35.381340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:35.400708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:35.401903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:35.469778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 493/991] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:35.741320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbf730-f5ee-415f-9310-1b86216528c0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:35.829709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:35.855946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbf730-f5ee-415f-9310-1b86216528c0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:35.890148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:35.890809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:35.891091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:35.891365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:35.891941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:35.893304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bea0976b-edc0-4fed-ba3b-a047b77a9f1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:35.904244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:35.905301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:35.906137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:35.910400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 499/992] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:35.912072 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:35.966235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9677d-8ce1-4263-80d3-768ed58c982b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:35.966718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9677d-8ce1-4263-80d3-768ed58c982b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:35.967158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9677d-8ce1-4263-80d3-768ed58c982b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:35.969158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 500/993] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:35.974690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:35.975170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:35.975604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:35.996480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:35.998371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:36.018737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2543a0b-a2ed-43d9-a3a2-e4c90700f455 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.033745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:36.054151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.054795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.055130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.055395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.056025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:36.056327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.057701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.063426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 494/994] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:36.069216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.092760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 501/995] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:36.173260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbc3a3-0e67-4dcf-bb8b-8ce3acf1efb3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.208084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.208696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.208977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.209247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.209872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:36.210173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.212209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.222020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 495/996] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:36.223125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.287451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:36.325313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e5df9-6d57-40b8-b58e-d5129135951c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.359409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.360023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.360305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.360577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.361274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] RBAC: Adding query filter params (interface=public, service_id=6b02a77eda5d4b589e8f091920892689, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:36.361581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.362984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.368442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.373374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 502/997] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/endpoints?service_id=6b02a77eda5d4b589e8f091920892689&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:36.374746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.414454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:36.478132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7f883-67c4-4867-bbd2-60dcc007cac9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.481713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82295584-ce29-4827-9bde-5a363862c26d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.510538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:36.515580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.516207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.516493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.516752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.517329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.518863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.518965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a96518-c207-4729-8994-c24ae177c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.519699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a96518-c207-4729-8994-c24ae177c52b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.519983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a96518-c207-4729-8994-c24ae177c52b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.520253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a96518-c207-4729-8994-c24ae177c52b None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.521396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a96518-c207-4729-8994-c24ae177c52b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.523483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a96518-c207-4729-8994-c24ae177c52b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:36.528186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:36.529156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:36.530158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 496/998] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:36.531938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 503/999] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:15:36.535144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.612373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a0ead-e6a1-4a31-bbc9-77fd784842de None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.612681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a0ead-e6a1-4a31-bbc9-77fd784842de None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.612848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a0ead-e6a1-4a31-bbc9-77fd784842de None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.614247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 504/1000] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:36.621691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.622121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.622433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.643997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:36.645246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:36.649829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e2163-cb41-43c4-9770-e512aa55c0b3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.657607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:36.687228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.687846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.688129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.688401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.689035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:36.689998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.691073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.699794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 497/1001] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:36.704209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.725041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 505/1002] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:36.812031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed913c46-2d45-411b-826f-1fbb086f402d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.849610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:36.850226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:36.850536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:36.850813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:36.851447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:36.851798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:36.853175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:36.862432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 498/1003] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:36.864374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:36.968408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578a965-da63-4c10-9e15-2509a05d7852 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:36.993397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:37.025518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:37.033131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:37.038056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:37.038704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:37.038988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:37.039308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:37.040122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:37.041510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:37.100177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b09ef6-bddc-44cf-9df9-83e7c635fa3a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:37.102570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 506/1004] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:37.149665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:37.150326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:37.150636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:37.150905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] PATH_INFO: `/v3/services/image` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:37.151505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:37.152929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:37.162644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:37.163825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:37.165317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:37.166876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 499/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Sep 3 01:15:36 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:37.170611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:37.277570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edeff4e-1d47-4487-abae-d5a7bec7b589 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:37.312280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:37.312890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:37.313166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:37.313433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:37.314056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] RBAC: Adding query filter params (name=image) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:37.314374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:37.315746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc24870-8ccf-4270-ab0d-d79947168721 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:37.321609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 507/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:37.327079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:37.422047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:37.428902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91162ab-837f-4708-b31f-21842982a286 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:37.463361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:37.463968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:37.464246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:37.464514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:37.465145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] RBAC: Adding query filter params (type=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:37.465443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:37.466847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c7758-f799-4dc1-baf7-d4b3588eab71 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:37.471429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 500/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:37 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:38.518443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19223375-0e75-40ce-88c2-2b6766b99032 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:38.518967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19223375-0e75-40ce-88c2-2b6766b99032 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:38.519073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19223375-0e75-40ce-88c2-2b6766b99032 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:38.520650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 508/1008] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:38.526718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627a28c-d9fb-4d88-9092-3df925fa9787 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:38.527226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627a28c-d9fb-4d88-9092-3df925fa9787 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:38.527303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627a28c-d9fb-4d88-9092-3df925fa9787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:38.546776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6627a28c-d9fb-4d88-9092-3df925fa9787 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:38.547989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6627a28c-d9fb-4d88-9092-3df925fa9787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:38.616740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 501/1009] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:38.894105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19223375-0e75-40ce-88c2-2b6766b99032 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:38.946864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:39.002210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19223375-0e75-40ce-88c2-2b6766b99032 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.047028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5b60-bd2e-455c-95f3-6fc23b13d8e4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.047333 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5b60-bd2e-455c-95f3-6fc23b13d8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.047576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de5b60-bd2e-455c-95f3-6fc23b13d8e4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.049064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 502/1010] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:39.053676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.053901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.054175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.059188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.059832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.060353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.060639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.062098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.063524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:39.072662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:39.073843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:39.078510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 03 01:15:39.079460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:39.080223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:39.080308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 509/1011] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:39.081084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:39.144240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 503/1012] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:39.187775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c872ce2-258a-4dfd-99f6-1fdcb5c8bcd8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.200358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:39.224484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.225095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.225373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.225696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.226329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:39.226692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.228127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dc943-955e-4000-b8b0-48e752f6496d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:39.234479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 510/1013] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:39.239123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:39.259845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:39.344985 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57733c90-eaa1-46e0-9c5d-2354386f7a99 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.367327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cd51-d9e5-40f5-bd09-afda2c3691cd None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.367687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cd51-d9e5-40f5-bd09-afda2c3691cd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.367945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cd51-d9e5-40f5-bd09-afda2c3691cd None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.369460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 511/1014] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:39.374016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.374318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.374593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.381121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.382590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.382897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.383173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.383815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:39.384124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.385514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:39.395841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 504/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Sep 3 01:15:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:39.396849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:39.397195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:39.398039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:39.468313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 512/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:39.469154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:39.482843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:39.501031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b42f9-cff4-4670-9ec1-cbac693b17eb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.542859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.543791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.544021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.544291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.545059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.546486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:39.576855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9678e0d-18b1-4d7f-8866-c50647181762 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.580036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:39.612377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.612992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.613270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.613558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.614666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.616222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:39.622413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 505/1017] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:39.622623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 513/1018] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:39.623717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:39.741444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce3196-9a57-4649-b6f4-5aa09db01d2b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.748874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:39.780428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247a09e-fead-4b0d-9e56-dc8885681a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.781331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247a09e-fead-4b0d-9e56-dc8885681a9d None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.781615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247a09e-fead-4b0d-9e56-dc8885681a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.781886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247a09e-fead-4b0d-9e56-dc8885681a9d None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.782565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247a09e-fead-4b0d-9e56-dc8885681a9d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.784022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247a09e-fead-4b0d-9e56-dc8885681a9d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:39.788801 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:39.851429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 506/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:39 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:39.869696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb475d-4e2f-43be-9d15-33ac7e8b5260 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:39.917591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9181d-5cc0-46ca-aabf-683d64d6ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:39.918532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9181d-5cc0-46ca-aabf-683d64d6ce5a None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:39.918818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9181d-5cc0-46ca-aabf-683d64d6ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:39.919088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9181d-5cc0-46ca-aabf-683d64d6ce5a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:39.919782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9181d-5cc0-46ca-aabf-683d64d6ce5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:39.921263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9181d-5cc0-46ca-aabf-683d64d6ce5a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:40.037303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 514/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:15:39 2022] POST /identity/v3/services => generated 242 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:15:41.671666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e943-3949-40e6-8848-58e0fb4c6efa None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:41.672202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e943-3949-40e6-8848-58e0fb4c6efa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:41.672202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a0e943-3949-40e6-8848-58e0fb4c6efa None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:41.673710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 507/1021] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:41.678596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42334aca-be28-4ced-a57d-47c124233906 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:41.679006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42334aca-be28-4ced-a57d-47c124233906 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:41.679170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42334aca-be28-4ced-a57d-47c124233906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:41.698128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-42334aca-be28-4ced-a57d-47c124233906 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:41.699341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-42334aca-be28-4ced-a57d-47c124233906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:41.724931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:41.772581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 515/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:41.837770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882e82e-e408-486c-9850-b5cf6a4a55a9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:41.838071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882e82e-e408-486c-9850-b5cf6a4a55a9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:41.838328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882e82e-e408-486c-9850-b5cf6a4a55a9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:41.839850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 508/1023] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:41.844871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be2f62-1669-4a19-9fae-ec532b4a1d03 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:41.845165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be2f62-1669-4a19-9fae-ec532b4a1d03 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:41.845506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be2f62-1669-4a19-9fae-ec532b4a1d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:41.865072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-84be2f62-1669-4a19-9fae-ec532b4a1d03 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:41.866278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-84be2f62-1669-4a19-9fae-ec532b4a1d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:41.937521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 516/1024] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:41.964469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6570-f50e-4807-95f1-a201a839d90f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:41.964790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6570-f50e-4807-95f1-a201a839d90f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:41.965023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa6570-f50e-4807-95f1-a201a839d90f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:41.970272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 509/1025] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:41.971866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43603909-c642-448c-841b-730f45d08b08 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:41.972164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43603909-c642-448c-841b-730f45d08b08 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:41.972445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43603909-c642-448c-841b-730f45d08b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:41.990946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-43603909-c642-448c-841b-730f45d08b08 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:41.992307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-43603909-c642-448c-841b-730f45d08b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:42.042136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6570-f50e-4807-95f1-a201a839d90f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.065695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 517/1026] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:42.150944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa6570-f50e-4807-95f1-a201a839d90f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.185611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.186229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.186528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.186798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.197512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:42.202211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:42.203148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:42.207287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 510/1027] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:42.209487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43603909-c642-448c-841b-730f45d08b08 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.215686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.233282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.272856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.325958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43603909-c642-448c-841b-730f45d08b08 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.330994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a883b4-5bd8-4c3d-9240-603757c0961c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.360841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.361448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.361725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.361994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.362702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:42.363007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.364364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.368347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.368984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.369272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.369542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.370685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.372115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.372246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 518/1028] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:42.373025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.374385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 511/1029] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:42.379367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.456165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.474523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a196b2-0ddb-4539-83cf-6565352ee723 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.477790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a7a52-4276-4012-9dbf-47d3f448da82 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.491972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.508577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.509203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.509482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.509751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.510321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.511715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.512187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.512940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.513272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.513900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:42.520771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:42.521963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:42.524425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 519/1030] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:42.525244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.525957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.527498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:42.551853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:42.553101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:42.554285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:42.554285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:42.554285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:42.554285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 512/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:42.554285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.629663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdbeed-b41a-4e4b-91ba-af805d43a1f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.652051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf274983-4bc2-472a-9dd1-f1799fdc1da7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.666209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.666866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.667144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.667411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.668049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.669475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.685160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.685763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.686043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.686312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.686963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] RBAC: Adding query filter params (name=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:42.687262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.688616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.693323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 513/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:42.694082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.723888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.771051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.776925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 520/1033] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:15:42 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:42.777789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.795735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06c8ad-2755-4547-8159-6bc49c369877 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.840549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.841179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.841462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.841737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.842442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:42.842752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.844186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.858717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 514/1034] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:42.864584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.900927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b04d9-0066-4388-84f3-c8f1db04e4a8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:42.942743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:42.943411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:42.944311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:42.944311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:42.944593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:42.944894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:42.946833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:42.954489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 521/1035] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:42.959771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:42.973191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:42.984242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dd874-113f-458c-8953-588109e0f412 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.024606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.025242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.025637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.025905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.033628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.035126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:43.036114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:43.048340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:43.049438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:43.050866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 515/1036] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:43.055139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:43.080499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f112d06-d0e6-4391-80d5-62e20dc497d4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.118704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.119316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.119599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.119870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.120571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] RBAC: Adding query filter params (interface=public, service_id=94bb4210677a4411858d5f81996f235f, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:43.120876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.122243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.131811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 522/1037] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:15:42 2022] GET /identity/v3/endpoints?service_id=94bb4210677a4411858d5f81996f235f&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:43.132868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:43.163317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a06429-11f3-47f0-ac92-3fc4994ffaf2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.202379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.203002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.203290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.203579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.204285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:43.204595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.206047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.225212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 516/1038] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:15:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:43.231279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:43.242642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:43.244052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240c939-a61e-4434-a902-a78ddeafd5a6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.272781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:43.282452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.283089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.283372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.283640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.284254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.285857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.301010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 523/1039] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:15:43 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:15:43.357379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ab496-7862-426d-bea0-297d100c9997 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.394057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.394722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.395007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.395280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.469436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.470968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.481497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:43.571653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 517/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 01:15:43.577101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:43.680298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183fb8b-05bc-4446-a40d-f0d4f6a6bad6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.713813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.714474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.714756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.715026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.725563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.727907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:43.742023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:43.743014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:43.743900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:43.743900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:43.743900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:43.743900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 524/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:43.749698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:43.859474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e56e21-25d9-4df2-a996-df001f6a053f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:43.869212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:43.893827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:43.894474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:43.894756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:43.895027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:43.895930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:43.896231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:43.897610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:43.910207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 518/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:43.915728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:43.967098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:44.023334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c96d9-4059-4949-bc26-aceaa70b090d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:44.058510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:44.059501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:44.059986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:44.060458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:44.067274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:44.068948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:44.080966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:44.081851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:44.082804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:44.086406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 525/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:44.089600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:44.194795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367e5ea-bafc-4179-a137-c26d525c7abc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:44.230130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:44.230770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:44.231055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:44.231324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:44.232015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:44.232318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:44.233822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a9f49-5f00-44cf-8b5f-0887b3a61a31 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:44.250728 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 519/1044] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:44.251988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:44.355365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bd664-1830-4957-b492-3c6842b68fec None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:44.357735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:44.390773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:44.391662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:44.392127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:44.392435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:44.393492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] RBAC: Adding query filter params (role.id=f4b01b45806047c39ce1eefde1394038, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=1749220d9e9547eba7ea035508c365bd) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:44.393819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:44.395335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8ddf6-73cb-4d4d-9bb1-587993ed5456 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:44.410452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 526/1045] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:44 2022] GET /identity/v3/role_assignments?user.id=1749220d9e9547eba7ea035508c365bd&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f4b01b45806047c39ce1eefde1394038 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:44.786441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d74b3-c9a4-44cb-90fc-b1f10d395134 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:44.786971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d74b3-c9a4-44cb-90fc-b1f10d395134 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:44.786971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d74b3-c9a4-44cb-90fc-b1f10d395134 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:44.788502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 520/1046] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:44.793702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce887f-48ac-49ba-b898-7baf4573f6c8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:44.793995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce887f-48ac-49ba-b898-7baf4573f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:44.794268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce887f-48ac-49ba-b898-7baf4573f6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:44.816644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-71ce887f-48ac-49ba-b898-7baf4573f6c8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:44.818162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-71ce887f-48ac-49ba-b898-7baf4573f6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:44.895215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 527/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:45.166841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d74b3-c9a4-44cb-90fc-b1f10d395134 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.195608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:45.269605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3e477-9840-4666-a7df-09fb4c919ecb None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.270118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3e477-9840-4666-a7df-09fb4c919ecb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.270118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3e477-9840-4666-a7df-09fb4c919ecb None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.271904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 528/1048] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:45.275262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d74b3-c9a4-44cb-90fc-b1f10d395134 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:45.277310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1dc0-6570-472f-b764-361960e74641 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.277590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1dc0-6570-472f-b764-361960e74641 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.277855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d1dc0-6570-472f-b764-361960e74641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.296055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-662d1dc0-6570-472f-b764-361960e74641 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:45.297233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-662d1dc0-6570-472f-b764-361960e74641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:45.312866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:45.313746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.314204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.314679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.325895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:45.330951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:45.331946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:45.332834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:45.334815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 521/1049] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:45.340891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.371959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 529/1050] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:45.434784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:45.446668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bf6f7-4965-4165-a4dc-a58d55072859 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:45.483578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:45.484194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.484475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.484745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.485410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:45.485713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:45.487111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:45.496110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 522/1051] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:45.501698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1dc0-6570-472f-b764-361960e74641 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.530113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:45.601884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d1dc0-6570-472f-b764-361960e74641 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:45.635586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:45.636219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.636733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.636866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] PATH_INFO: `/v3/users/dfdfca08d5d641bb8e3dc071df71f71a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.637720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.651911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] RBAC: Authorizing `identity:get_user(user_id=dfdfca08d5d641bb8e3dc071df71f71a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:45.653361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:45.655183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 530/1052] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/users/dfdfca08d5d641bb8e3dc071df71f71a => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:45.660868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.746143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221859d4-a451-42af-a599-33508f04cff5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:45.750250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:45.766436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe6cac-23a8-44fd-8ecc-0b5291521383 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:45.781857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:45.782528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.782814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.783088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] PATH_INFO: `/v3/services/image` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.783699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:45.785078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:45.793868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:45.795276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:45.796124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 523/1053] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:45.800866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.811216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:45.812203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.812630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.813041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.817633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:45.890838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:45.893023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:45.913551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f03cb8-bb5a-4388-8409-9504255ac993 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:45.950412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:45.951302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:45.951752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:45.952022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:45.952697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] RBAC: Adding query filter params (name=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:45.952996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:45.954497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:45.959756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 524/1054] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:45.965626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:45.994627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:46.014369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 531/1055] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 01:15:46.017902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:46.059289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:46.084285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2ea14-e304-465a-9dce-763d53e6883c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:46.121890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:46.122530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.122844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.123068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.123691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] RBAC: Adding query filter params (type=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:46.123989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:46.124641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2e7ba-ad12-4d75-9604-b55132b47d33 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:46.125362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:46.136246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 525/1056] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:46.137226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:46.167164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:46.168108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.168580 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.168854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.169988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] RBAC: Authorizing `identity:get_project(project_id=de35c42f4b7547148100f3ac720b8305)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:46.171510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:46.173327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 532/1057] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:15:46 2022] GET /identity/v3/projects/de35c42f4b7547148100f3ac720b8305 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:46.179149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:46.241464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:46.255154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75352df-c988-4896-9ea4-f46614259ddf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:46.288319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48542d-d33c-456b-bb65-0d136e4f25f6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:46.294461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:46.295080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.295363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.295635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.296307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:46.297697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:46.324234 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:46.325024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.325304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.325568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.326290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=de35c42f4b7547148100f3ac720b8305, user.id=dfdfca08d5d641bb8e3dc071df71f71a) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:46.326619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:46.328032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafecc1-94c7-45b9-a698-c8a614407669 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:46.333552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 533/1058] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:46 2022] GET /identity/v3/role_assignments?user.id=dfdfca08d5d641bb8e3dc071df71f71a&scope.project.id=de35c42f4b7547148100f3ac720b8305&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:46.335472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23719961-992b-4c64-9455-b01fe4b43c40 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.336423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23719961-992b-4c64-9455-b01fe4b43c40 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.336763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23719961-992b-4c64-9455-b01fe4b43c40 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.338023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 534/1059] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:46.343593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.344009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.344188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.358753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:46.364602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:46.374373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:46.374605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 526/1060] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:15:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:46.459174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 535/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:46.732888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:46.845244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40694a92-09fd-49bf-aba5-261d385ebce5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:46.879715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:46.880508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:46.880882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:46.881254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:46.891703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:46.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:46.898997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:46.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:46.901762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 527/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Sep 3 01:15:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:46.907604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:46.955852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:47.012023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d42f5a-40e8-47f4-8764-bf8c69ac4b21 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:47.046792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:47.047401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:47.047681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:47.047947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:47.048608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:47.048907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:47.050281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:47.062871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 536/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:15:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:47.064010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:47.121760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:47.163498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70b883-8da0-42eb-adbf-2f074f9689ec None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:47.197142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:47.197772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:47.198058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:47.198331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:47.208808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:47.210255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:47.224188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:47.225172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:47.231675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:47.231675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:47.231675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:47.231675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 528/1064] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:47.233479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:47.335137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef2edf-b1a6-4686-8f90-76256ad975ee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:47.342440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:47.372185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:47.372801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:47.373080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:47.373349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:47.374015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:47.374316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:47.375702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:47.387504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 537/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:47.392923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:47.498921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4d035-8107-4dfd-ae39-f5bea4ca251c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:47.509919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:47.534877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:47.535700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:47.536134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:47.536555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:47.544725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:47.546379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:47.558488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:47.559619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:47.560806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:47.562929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 529/1066] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:15:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:47.568335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:47.668756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8ef0-0fdd-478b-a400-b373ac734bae None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:47.702397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:47.703144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:47.703518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:47.703884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:47.704726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:47.705123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:47.706660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:47.729979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 538/1067] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:15:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:47.731260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:47.842935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a6816-f6da-431e-8cbe-98ab66ff3bb2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:47.877633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:47.878280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:47.878571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:47.878849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:47.890458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:47.891937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:47.907127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:47.908114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:47.909005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:47.909005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:47.909005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:47.909767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 530/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:47.914119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:47.940816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:48.020407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae5ec9-bb3d-4122-b18d-22ba31044735 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.056712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.057313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.057588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.057855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.058541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:48.058844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:48.060198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:48.076465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 539/1069] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:48.077965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.104464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:48.189497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af2a5-74a5-456c-b872-df8a3111357b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.223287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.223879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.224163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.224436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.231842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:48.233291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-362c41c3-c510-47c5-859d-2da3f3658323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:48.244529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:48.245410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:48.246865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:48.247584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 531/1070] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:48.251589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.256878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6100a3-ef35-41dc-913c-c8874e0f12a5 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.257160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6100a3-ef35-41dc-913c-c8874e0f12a5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.258018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6100a3-ef35-41dc-913c-c8874e0f12a5 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.259739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 532/1071] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:48.264568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7848-737b-4b05-a099-2591ac4a8dfc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.264857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7848-737b-4b05-a099-2591ac4a8dfc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.265131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7848-737b-4b05-a099-2591ac4a8dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.284663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-039f7848-737b-4b05-a099-2591ac4a8dfc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:48.285885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-039f7848-737b-4b05-a099-2591ac4a8dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:48.343017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:48.356495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 533/1072] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:48.357816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53cc2-c07c-4a42-915c-4f67e9492194 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.358101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53cc2-c07c-4a42-915c-4f67e9492194 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.358422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c53cc2-c07c-4a42-915c-4f67e9492194 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.358792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f550d-be1f-41f4-9bf2-a4eff0af2ad3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.360234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 534/1073] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:48.368818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.369099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.369371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.392685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:48.394129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:48.397042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.397653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.397934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.398471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.399240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:48.399547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:48.400976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:48.403634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:48.413570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 540/1074] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:48.422370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.473639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 535/1075] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:48.522232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35ced1-4521-474e-a1a3-ce07806d0fc3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.557023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.557671 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.557952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.558222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/1749220d9e9547eba7ea035508c365bd/roles/f4b01b45806047c39ce1eefde1394038` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.560416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=1749220d9e9547eba7ea035508c365bd, role_id=f4b01b45806047c39ce1eefde1394038)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:48.561843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:48.626688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 541/1076] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:15:48 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/1749220d9e9547eba7ea035508c365bd/roles/f4b01b45806047c39ce1eefde1394038 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:15:48.626936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.737086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559fa06b-6929-4959-9d88-21675e2bc85d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.747225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.751231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:48.783220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.783865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.784145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.784416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.796545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:48.801408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:48.802629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:48.804015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 536/1077] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:48.809098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.812591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:48.870430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7bb0-1f0a-4048-839e-e874672df7eb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.907450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.908058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.908338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.908719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.926148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94370355-7fa6-4a1e-98b6-c48043edb7ae None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:48.964396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:48.965012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:48.965385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:48.965665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:48.966331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:48.966657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:48.968107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:48.977601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 537/1078] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:48.983221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:48.990048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:48.990048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:48.993363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:49.050284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:49.094418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe152b2-f9aa-4b5c-860f-611062c7bff1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:49.110405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 542/1079] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 01:15:49.111254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:49.130905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:49.131541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:49.131821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:49.132101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] PATH_INFO: `/v3/users/dfdfca08d5d641bb8e3dc071df71f71a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:49.133902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] RBAC: Authorizing `identity:get_user(user_id=dfdfca08d5d641bb8e3dc071df71f71a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:49.135383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:49.141481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 538/1080] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:15:48 2022] GET /identity/v3/users/dfdfca08d5d641bb8e3dc071df71f71a => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:49.142501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:49.216348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3bae5-ff4e-411c-96e9-0a911e8d4df5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:49.218747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:49.249673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6766cd-a93f-4ee6-a94b-cb765da3f988 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:49.252616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:49.253367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:49.253689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:49.253918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:49.267362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:49.269369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:49.284984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:49.286006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:49.286964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:49.286964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:49.286964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:49.286964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 543/1081] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:49.291962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:49.292617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:49.297567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:49.298614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:49.299089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:49.299524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:49.300903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] RBAC: Authorizing `identity:get_project(project_id=de35c42f4b7547148100f3ac720b8305)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:49.302757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:49.310140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 539/1082] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:15:49 2022] GET /identity/v3/projects/de35c42f4b7547148100f3ac720b8305 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:49.311697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:49.401985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cddd6a-2097-42cd-971b-b2b71494d3ec None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:49.415493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b7f891-ef24-4976-9918-2d762cf9bde6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:49.441159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:49.442061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:49.442376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:49.442913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:49.443567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:49.444016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:49.446120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df008f-7705-4612-8c33-8bbb8b344af3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:49.449330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419c21d-7cad-4102-9bb2-50be73b4aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:49.449979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419c21d-7cad-4102-9bb2-50be73b4aa44 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:49.450257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419c21d-7cad-4102-9bb2-50be73b4aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:49.450555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419c21d-7cad-4102-9bb2-50be73b4aa44 None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305/users/dfdfca08d5d641bb8e3dc071df71f71a/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:49.452458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419c21d-7cad-4102-9bb2-50be73b4aa44 None admin] RBAC: Authorizing `identity:create_grant(project_id=de35c42f4b7547148100f3ac720b8305, user_id=dfdfca08d5d641bb8e3dc071df71f71a, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:49.453884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419c21d-7cad-4102-9bb2-50be73b4aa44 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:49.458946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 544/1083] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:49.519453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 540/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:15:49 2022] PUT /identity/v3/projects/de35c42f4b7547148100f3ac720b8305/users/dfdfca08d5d641bb8e3dc071df71f71a/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:15:50.569496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099fb53-2110-4c4e-bdfa-a0ebd2db1eb3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:50.570072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099fb53-2110-4c4e-bdfa-a0ebd2db1eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:50.570162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099fb53-2110-4c4e-bdfa-a0ebd2db1eb3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:50.571701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 545/1085] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:50.577014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:50.577445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:50.577551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:50.597182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:50.598445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:50.654399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:50.665850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 541/1086] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:50.933536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099fb53-2110-4c4e-bdfa-a0ebd2db1eb3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:50.951996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:51.036092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099fb53-2110-4c4e-bdfa-a0ebd2db1eb3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.068953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.069582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.070418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.070697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.080784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:51.086244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:51.087427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:51.088290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 546/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Sep 3 01:15:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:15:51.093421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.190399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874c7be-13d0-4768-8fcc-e46f8a8c04ca None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.224309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.224915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.225195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.225470 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.226169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:51.226581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:51.227966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:51.235677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 542/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:51.241235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.318655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:51.341963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d4b4b-b935-4735-83c4-3e48657772bd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.376361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.376989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.377269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.377539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.388415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:51.389860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e6d8e6ff-4cad-496e-a136-e67ff16982d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:51.405696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:51.411623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:51.412597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:51.412597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:51.412597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:51.412597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 547/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:51.412842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.423865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6409dae-8cf1-4d47-b3fd-08675fb31756 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.424172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6409dae-8cf1-4d47-b3fd-08675fb31756 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.424501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6409dae-8cf1-4d47-b3fd-08675fb31756 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.425818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 548/1090] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:15:51.431102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.431383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.431655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.449767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:51.450987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:51.466049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:51.514529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407aa22-b052-4037-9b82-11722403bbf9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.518466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 549/1091] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:15:51.547382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.547993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.548277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.548548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.549192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:51.549497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:51.550909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:51.562923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 543/1092] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:51.568613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.625779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:51.667748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c51f6-b74f-429f-9e41-c30d5af34d87 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.701001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.701558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.701838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.702106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.708884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:51.710325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:15:51.721537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:51.722584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:51.723953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 550/1093] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:51.729038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.787874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.828595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3bc4-f34d-4d4c-9909-c52ba910918e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.842941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:51.863146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.863748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.864029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.864294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.865040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:51.865343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:51.866855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:51.884972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 544/1094] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:51.890803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.891223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e2d8d-e435-4e8b-ae7c-0ec85f9b31ed None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:51.908366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:51.925596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:51.926226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:51.926546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:51.926821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:51.937679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:15:51.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:51.943465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:15:51.945403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 551/1095] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:51.950280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:51.994212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad7c45-6197-417a-b672-b2e2829f462a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.028637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.029250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.029527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.029803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.068380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.078435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769b4f2-92fc-4a0c-b886-c96374f33c30 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.096462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.097921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.111174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.111776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.112080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.112352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.113006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:52.113307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.114701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.123450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 552/1096] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:52.129318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.171382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.204202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 545/1097] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 01:15:52.209975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.230005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9de76-6360-4294-9c06-8b0b4e2ac40e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.264342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.264972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.265251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.265521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] PATH_INFO: `/v3/users/dfdfca08d5d641bb8e3dc071df71f71a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.266696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=dfdfca08d5d641bb8e3dc071df71f71a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.268123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.269849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 553/1098] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/users/dfdfca08d5d641bb8e3dc071df71f71a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:52.276504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.276942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.308482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24265e98-2d29-4313-9925-eb9da0d3d412 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.341612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.342236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.342538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.342809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.354252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.355747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:15:52.369883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:52.371116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:15:52.372224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:15:52.372224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:15:52.372224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:52.372224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 546/1099] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:52.375498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d070a08-9ae7-46b6-b831-14eaa773e8a6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.376518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.385539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.410845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.411542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.411822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.412092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.475289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e515fc-933f-40ad-a38a-351c77abee54 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.480121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.481577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.496793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.509231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.509839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.510121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.510418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.511149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:52.511451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.512832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.530503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 547/1100] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:52.531399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.579741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 554/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 03 01:15:52.586205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.595470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.631359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cc08b-5956-488f-9720-2c77ba490f27 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.667760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.668465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.668747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.669018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.675523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.676976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.685125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e24a1-245d-4a9e-82f3-371c6a9c9e1b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:15:52.688187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:15:52.689231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:15:52.690365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:15:52.694415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 548/1102] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:15:52.695841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.720978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.721613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.721894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.722165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.723299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] RBAC: Authorizing `identity:get_project(project_id=de35c42f4b7547148100f3ac720b8305)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.724728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.726421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 555/1103] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/projects/de35c42f4b7547148100f3ac720b8305 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:52.732322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.738368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.794758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25927c-03cd-450c-8513-0154b26222b7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.809140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.830220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.830887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.831131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.831448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.832090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:52.832393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.832743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ada52-6e0d-410c-a7ca-03abd0122050 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.833865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.846253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 549/1104] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:52.852758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:52.869117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.869724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.870010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.870284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.871046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=de35c42f4b7547148100f3ac720b8305, user.id=dfdfca08d5d641bb8e3dc071df71f71a) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:52.871355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.872786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f16dc-c3b7-4fa1-b91d-29d9bfdc08ff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:52.890383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 556/1105] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/role_assignments?user.id=dfdfca08d5d641bb8e3dc071df71f71a&scope.project.id=de35c42f4b7547148100f3ac720b8305&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:52.956658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb64d2a-7a40-4c06-a301-f31c1ade88ea None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:52.958360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:52.993938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:52.994584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:52.994868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:52.995140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:52.995881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] RBAC: Adding query filter params (role.id=f4b01b45806047c39ce1eefde1394038, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=1749220d9e9547eba7ea035508c365bd) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:52.996185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:52.997655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729a6ff-ca57-43c9-b65c-858cbb631e64 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:53.004049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 550/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:52 2022] GET /identity/v3/role_assignments?user.id=1749220d9e9547eba7ea035508c365bd&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f4b01b45806047c39ce1eefde1394038 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:15:54.811276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06c55-0020-4fcf-9034-226b7d47cea7 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:54.811688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06c55-0020-4fcf-9034-226b7d47cea7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:54.811820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae06c55-0020-4fcf-9034-226b7d47cea7 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:54.813167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 557/1107] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:54.818403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:54.818840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:54.818952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:54.838113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:54.839336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:54.903716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 551/1108] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:55.176594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae06c55-0020-4fcf-9034-226b7d47cea7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:55.276353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae06c55-0020-4fcf-9034-226b7d47cea7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:55.312043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf9cff-8c79-4d23-9d19-a7ba204408fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:55.312680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf9cff-8c79-4d23-9d19-a7ba204408fb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:55.312958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf9cff-8c79-4d23-9d19-a7ba204408fb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:55.313225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf9cff-8c79-4d23-9d19-a7ba204408fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:55.314363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf9cff-8c79-4d23-9d19-a7ba204408fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:55.315801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf9cff-8c79-4d23-9d19-a7ba204408fb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:55.317377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 558/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:15:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:55.323848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:55.414688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:55.420237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfaa6d-9996-49f3-9c45-d91c9b58b1f0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:55.453258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcdd35-91b7-4bf0-9c14-80475b6cdaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:55.453786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdd35-91b7-4bf0-9c14-80475b6cdaf2 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:55.454064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdd35-91b7-4bf0-9c14-80475b6cdaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:55.454331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcdd35-91b7-4bf0-9c14-80475b6cdaf2 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:55.455001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdd35-91b7-4bf0-9c14-80475b6cdaf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:55.456430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcdd35-91b7-4bf0-9c14-80475b6cdaf2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:55.539917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 552/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:15:55 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:15:57.479800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdf645-4f8e-4e80-8b82-db75640cbaba None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:57.480374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdf645-4f8e-4e80-8b82-db75640cbaba None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:57.480374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fdf645-4f8e-4e80-8b82-db75640cbaba None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:57.481695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 559/1111] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:15:57.486916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:57.487288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:57.487469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:57.505275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:15:57.506488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:15:57.571319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 553/1112] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:15:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:15:57.842094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdf645-4f8e-4e80-8b82-db75640cbaba None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:57.853052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:57.941901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fdf645-4f8e-4e80-8b82-db75640cbaba None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:57.976963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:57.977593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:57.977872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:57.978141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] PATH_INFO: `/v3/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:57.979588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] RBAC: Authorizing `identity:get_role(role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:57.980958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:57.982299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 560/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:15:57 2022] GET /identity/v3/roles/3a297fae4df9419b9c53354550781a57 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:57.988136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:58.068119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:58.085944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af450ca-d4a0-4305-a227-b58bdf3e0079 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:58.119327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:58.119942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:58.120221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:58.120491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] PATH_INFO: `/v3/users/7b2ea8b3f5134ac88ec5849b0c7b602f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:58.133210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] RBAC: Authorizing `identity:get_user(user_id=7b2ea8b3f5134ac88ec5849b0c7b602f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:58.134740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:58.136323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 554/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:15:57 2022] GET /identity/v3/users/7b2ea8b3f5134ac88ec5849b0c7b602f => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:58.142264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:58.222186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:58.245806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4189150-bf76-4f91-847f-e6cb578adcc7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:58.279712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:58.280327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:58.280607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:58.280969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:58.370397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:58.371777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:58.444235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:15:58.467661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 561/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:15:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:15:58.473602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:58.568366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c0441-e87b-4548-930c-da7dfb038a6f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:58.602393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cea74c-b449-49c0-9102-cbf2ea44d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:58.603010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cea74c-b449-49c0-9102-cbf2ea44d5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:58.603287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cea74c-b449-49c0-9102-cbf2ea44d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:58.603555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cea74c-b449-49c0-9102-cbf2ea44d5f4 None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:58.604648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cea74c-b449-49c0-9102-cbf2ea44d5f4 None admin] RBAC: Authorizing `identity:get_project(project_id=de35c42f4b7547148100f3ac720b8305)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:58.606078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cea74c-b449-49c0-9102-cbf2ea44d5f4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:58.607713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 555/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:15:58 2022] GET /identity/v3/projects/de35c42f4b7547148100f3ac720b8305 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:15:58.613807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:15:58.709615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec747adb-b6eb-4f4e-a484-86c9a028f1c9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:15:58.742869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:15:58.743459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:15:58.743734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:15:58.744001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:15:58.744715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] RBAC: Adding query filter params (role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=de35c42f4b7547148100f3ac720b8305, user.id=7b2ea8b3f5134ac88ec5849b0c7b602f) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:15:58.745020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:15:58.746461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f65da-ca9f-4f53-9a76-dfb820c2f12b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:15:58.751667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 562/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:15:58 2022] GET /identity/v3/role_assignments?user.id=7b2ea8b3f5134ac88ec5849b0c7b602f&scope.project.id=de35c42f4b7547148100f3ac720b8305&role.id=3a297fae4df9419b9c53354550781a57 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:16:00.720688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d4a42-184b-4f35-a698-79adc0442153 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:00.721289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d4a42-184b-4f35-a698-79adc0442153 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:00.721289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d4a42-184b-4f35-a698-79adc0442153 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:00.722641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 556/1118] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:00.730096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:00.730601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:00.730702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:00.749921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:00.751173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:00.789978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:00.825468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 563/1119] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:01.107006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d4a42-184b-4f35-a698-79adc0442153 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:01.118191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:01.207867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d4a42-184b-4f35-a698-79adc0442153 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:01.241917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:01.242642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:01.242926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:01.243200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] PATH_INFO: `/v3/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:01.244584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] RBAC: Authorizing `identity:get_role(role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:01.245962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:01.247321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 557/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:01 2022] GET /identity/v3/roles/3a297fae4df9419b9c53354550781a57 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:01.253147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:01.360351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b166-3d57-44bd-b633-b7a0d30c4c39 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:01.392336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:01.392951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:01.393233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:01.393498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] PATH_INFO: `/v3/users/7b2ea8b3f5134ac88ec5849b0c7b602f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:01.394615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] RBAC: Authorizing `identity:get_user(user_id=7b2ea8b3f5134ac88ec5849b0c7b602f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:01.396295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:01.398432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 564/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:01 2022] GET /identity/v3/users/7b2ea8b3f5134ac88ec5849b0c7b602f => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:01.404407 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:01.479203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:01.500721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3ff1c-31fa-4b19-864a-4bc2b382a640 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:01.534645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0424cf2-7a44-41e6-9029-cb0ba52b7b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:01.535259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0424cf2-7a44-41e6-9029-cb0ba52b7b3e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:01.535538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0424cf2-7a44-41e6-9029-cb0ba52b7b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:01.535810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0424cf2-7a44-41e6-9029-cb0ba52b7b3e None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:01.536909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0424cf2-7a44-41e6-9029-cb0ba52b7b3e None admin] RBAC: Authorizing `identity:get_project(project_id=de35c42f4b7547148100f3ac720b8305)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:01.539070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0424cf2-7a44-41e6-9029-cb0ba52b7b3e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:01.540668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 558/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:01 2022] GET /identity/v3/projects/de35c42f4b7547148100f3ac720b8305 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:01.546528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:01.588787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:01.653000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20889a6-0c7d-4e86-8536-3fef9b2c3e7f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:01.686087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4c793-065c-443c-bdb7-db795d5aa838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:01.686762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4c793-065c-443c-bdb7-db795d5aa838 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:01.687040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4c793-065c-443c-bdb7-db795d5aa838 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:01.687307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4c793-065c-443c-bdb7-db795d5aa838 None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305/users/7b2ea8b3f5134ac88ec5849b0c7b602f/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:01.689392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4c793-065c-443c-bdb7-db795d5aa838 None admin] RBAC: Authorizing `identity:create_grant(project_id=de35c42f4b7547148100f3ac720b8305, user_id=7b2ea8b3f5134ac88ec5849b0c7b602f, role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:01.690858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4c793-065c-443c-bdb7-db795d5aa838 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:01.757215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 565/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:16:01 2022] PUT /identity/v3/projects/de35c42f4b7547148100f3ac720b8305/users/7b2ea8b3f5134ac88ec5849b0c7b602f/roles/3a297fae4df9419b9c53354550781a57 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:16:03.638932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297de597-7eaf-45ba-9886-1e00af8bcaf2 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:03.639488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297de597-7eaf-45ba-9886-1e00af8bcaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:03.639488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297de597-7eaf-45ba-9886-1e00af8bcaf2 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:03.640809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 559/1124] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:03.646284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:03.646720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:03.646871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:03.665937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:03.667157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:03.732335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 566/1125] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:04.004319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297de597-7eaf-45ba-9886-1e00af8bcaf2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:04.103981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297de597-7eaf-45ba-9886-1e00af8bcaf2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:04.136549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:04.137166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:04.137443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:04.137714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] PATH_INFO: `/v3/roles/3a297fae4df9419b9c53354550781a57` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:04.139147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] RBAC: Authorizing `identity:get_role(role_id=3a297fae4df9419b9c53354550781a57)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:04.140510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:04.141797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 560/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:04 2022] GET /identity/v3/roles/3a297fae4df9419b9c53354550781a57 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:04.148398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:04.175845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:04.251732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058f4e8-83ad-4d48-a125-24c5dfa3d431 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:04.287057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:04.287608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:04.287891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:04.288736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] PATH_INFO: `/v3/users/7b2ea8b3f5134ac88ec5849b0c7b602f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:04.289859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] RBAC: Authorizing `identity:get_user(user_id=7b2ea8b3f5134ac88ec5849b0c7b602f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:04.291327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:04.292930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 567/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:04 2022] GET /identity/v3/users/7b2ea8b3f5134ac88ec5849b0c7b602f => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:04.298909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:04.313263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:04.395159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20883589-3ba6-41da-bf8d-6374d8ab3f8d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:04.428287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:04.429062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:04.429343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:04.429608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:04.493596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:04.495062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:04.508130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:04.590856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 561/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:16:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:16:04.596717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:04.689458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:04.695666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a623253-bda6-4310-a976-d543f1cd9946 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:04.728194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd5393-48eb-4bfd-acca-6dbbcec32e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:04.729060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5393-48eb-4bfd-acca-6dbbcec32e7d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:04.729520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5393-48eb-4bfd-acca-6dbbcec32e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:04.730114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd5393-48eb-4bfd-acca-6dbbcec32e7d None admin] PATH_INFO: `/v3/projects/de35c42f4b7547148100f3ac720b8305` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:04.731258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd5393-48eb-4bfd-acca-6dbbcec32e7d None admin] RBAC: Authorizing `identity:get_project(project_id=de35c42f4b7547148100f3ac720b8305)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:04.732690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd5393-48eb-4bfd-acca-6dbbcec32e7d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:04.734319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 568/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:04 2022] GET /identity/v3/projects/de35c42f4b7547148100f3ac720b8305 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:04.740729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:04.838695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a654-9a35-4c26-8619-3497aae7467f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:04.851403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:04.872155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:04.872760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:04.873043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:04.873317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:04.874048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] RBAC: Adding query filter params (role.id=3a297fae4df9419b9c53354550781a57, scope.project.id=de35c42f4b7547148100f3ac720b8305, user.id=7b2ea8b3f5134ac88ec5849b0c7b602f) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:04.874376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:04.875859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6b998-3e97-4d17-aa80-0c8940db0dc5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:04.881403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 562/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:16:04 2022] GET /identity/v3/role_assignments?user.id=7b2ea8b3f5134ac88ec5849b0c7b602f&scope.project.id=de35c42f4b7547148100f3ac720b8305&role.id=3a297fae4df9419b9c53354550781a57 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:06.809303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4389d9-72ee-44af-b083-f586acaa1c5e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:06.809721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4389d9-72ee-44af-b083-f586acaa1c5e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:06.809846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4389d9-72ee-44af-b083-f586acaa1c5e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:06.811241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 569/1131] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:06.816388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:06.816802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:06.816933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:06.836098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:06.837276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:06.900807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 563/1132] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:07.168234 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4389d9-72ee-44af-b083-f586acaa1c5e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:07.267047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4389d9-72ee-44af-b083-f586acaa1c5e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:07.299449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cc69a-671f-4055-bfa7-233d29afb257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:07.300061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cc69a-671f-4055-bfa7-233d29afb257 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:07.300335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cc69a-671f-4055-bfa7-233d29afb257 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:07.300600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cc69a-671f-4055-bfa7-233d29afb257 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:07.301687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cc69a-671f-4055-bfa7-233d29afb257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:07.303139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cc69a-671f-4055-bfa7-233d29afb257 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:07.305240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 570/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:16:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:07.311007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:07.406394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b0bed-fb18-4924-9132-4cb3cd84b4fb None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:07.413348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:07.438219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43679caa-313d-47c1-a2b4-1e353979043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:07.438779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679caa-313d-47c1-a2b4-1e353979043b None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:07.439057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679caa-313d-47c1-a2b4-1e353979043b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:07.439325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43679caa-313d-47c1-a2b4-1e353979043b None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:07.439959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43679caa-313d-47c1-a2b4-1e353979043b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:07.441385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43679caa-313d-47c1-a2b4-1e353979043b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:07.529337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 564/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:16:07 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:16:09.438063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6fa2b-fb23-4ca7-8707-8a62397cad3c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:09.438651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6fa2b-fb23-4ca7-8707-8a62397cad3c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:09.438651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6fa2b-fb23-4ca7-8707-8a62397cad3c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:09.439988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 571/1135] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:09.446125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:09.446568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:09.446699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:09.464713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:09.465880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:09.529361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 565/1136] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:09.796972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6fa2b-fb23-4ca7-8707-8a62397cad3c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:09.828192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:09.898382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6fa2b-fb23-4ca7-8707-8a62397cad3c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:09.930712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146facd-7088-4dfc-b78f-d7d63fdc83de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:09.931330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146facd-7088-4dfc-b78f-d7d63fdc83de None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:09.931608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146facd-7088-4dfc-b78f-d7d63fdc83de None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:09.931874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146facd-7088-4dfc-b78f-d7d63fdc83de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:09.932967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146facd-7088-4dfc-b78f-d7d63fdc83de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:09.934418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146facd-7088-4dfc-b78f-d7d63fdc83de None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:09.935982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 572/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:16:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:09.941649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:10.030019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:10.041588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8931a-9587-4399-beac-30cc4fdb1354 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:10.075455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce68a81-e50f-4e82-8c4c-eede87db4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:10.075997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce68a81-e50f-4e82-8c4c-eede87db4ecd None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:10.076279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce68a81-e50f-4e82-8c4c-eede87db4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:10.076549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce68a81-e50f-4e82-8c4c-eede87db4ecd None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:10.077188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce68a81-e50f-4e82-8c4c-eede87db4ecd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:10.078653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce68a81-e50f-4e82-8c4c-eede87db4ecd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:10.180398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 566/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:16:09 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:16:12.099798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d6435-feeb-4995-be91-440b6ec336f6 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:12.100814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d6435-feeb-4995-be91-440b6ec336f6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:12.100909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d6435-feeb-4995-be91-440b6ec336f6 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:12.102223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 573/1139] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:12.107275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:12.107676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:12.107826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:12.126261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:12.127460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:12.190971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 567/1140] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:12.457222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d6435-feeb-4995-be91-440b6ec336f6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:12.555074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:12.558796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d6435-feeb-4995-be91-440b6ec336f6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:12.591438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:12.592054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:12.592334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:12.592603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:12.603240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:16:12.608297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:12.609478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:16:12.610630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 574/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:12.615508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:12.694836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:12.719088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e09ae-1bf2-486f-865e-7f594a4d1ed2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:12.752335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:12.752930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:12.753206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:12.753473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:12.754129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:12.754454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:12.755816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:12.764050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 568/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:16:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:12.769859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:12.867604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfb875-5c65-456d-8a5d-b4a61933b5df None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:12.900517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ff152-7781-4e62-a689-91b592010aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:12.901132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ff152-7781-4e62-a689-91b592010aee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:12.901411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ff152-7781-4e62-a689-91b592010aee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:12.901689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ff152-7781-4e62-a689-91b592010aee None admin] PATH_INFO: `/v3/users/f36035b9604948bf8e5c61d1c125cfbb` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:12.914757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ff152-7781-4e62-a689-91b592010aee None admin] RBAC: Authorizing `identity:get_user(user_id=f36035b9604948bf8e5c61d1c125cfbb)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:12.916284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ff152-7781-4e62-a689-91b592010aee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:12.917868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 575/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:12 2022] GET /identity/v3/users/f36035b9604948bf8e5c61d1c125cfbb => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:12.923719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:13.024686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cacf6d-82f9-46f2-ba50-c1f90b909e91 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:13.057290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:13.057888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:13.058163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:13.058455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:13.061739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:13.125154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:13.126683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:13.221509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 569/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:16:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:16:13.228126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ff152-7781-4e62-a689-91b592010aee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:13.246797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:13.329720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ff152-7781-4e62-a689-91b592010aee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:13.362985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd137f01-0395-4c91-a2e2-8366655175a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:13.363611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd137f01-0395-4c91-a2e2-8366655175a4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:13.363891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd137f01-0395-4c91-a2e2-8366655175a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:13.364158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd137f01-0395-4c91-a2e2-8366655175a4 None admin] PATH_INFO: `/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:13.365261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd137f01-0395-4c91-a2e2-8366655175a4 None admin] RBAC: Authorizing `identity:get_project(project_id=67fdc3ebe0024d54829c17e6da4ceb24)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:13.366773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd137f01-0395-4c91-a2e2-8366655175a4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:13.368404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 576/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:13 2022] GET /identity/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:13.374618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:13.407993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:13.470468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723992-fbdc-4d14-bf3a-f47307629d7b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:13.502063 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:13.502762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:13.503044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:13.503311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:13.504021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=67fdc3ebe0024d54829c17e6da4ceb24, user.id=f36035b9604948bf8e5c61d1c125cfbb) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:13.504321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:13.505740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cabff3-e56e-4497-b541-f147955a0836 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:13.511572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 570/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:16:13 2022] GET /identity/v3/role_assignments?user.id=f36035b9604948bf8e5c61d1c125cfbb&scope.project.id=67fdc3ebe0024d54829c17e6da4ceb24&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:15.379875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f648b-c5b4-431a-86a8-6bb891e97ad9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:15.380403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f648b-c5b4-431a-86a8-6bb891e97ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:15.380403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f648b-c5b4-431a-86a8-6bb891e97ad9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:15.381921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 577/1147] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:15.387134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:15.387553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:15.387686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:15.406165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:15.407381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:15.471743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 571/1148] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:15.743826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f648b-c5b4-431a-86a8-6bb891e97ad9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:15.827775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:15.843073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f648b-c5b4-431a-86a8-6bb891e97ad9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:15.876783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:15.877405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:15.877683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:15.877951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:15.888033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:16:15.892845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:15.893982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:16:15.895094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 578/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:15.900382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:15.929822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:15.996018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e48ac7-05ac-43fa-9ca2-e929271f2649 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:16.027984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:16.028578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:16.028855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:16.029123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:16.029790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:16.030068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:16.031460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:16.039751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 572/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:16.045545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:16.144058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6274f3e-bbca-40c2-a186-32a2807edf39 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:16.177101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:16.177705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:16.177982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:16.178297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] PATH_INFO: `/v3/users/f36035b9604948bf8e5c61d1c125cfbb` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:16.179416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] RBAC: Authorizing `identity:get_user(user_id=f36035b9604948bf8e5c61d1c125cfbb)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:16.180829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:16.182385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 579/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:16 2022] GET /identity/v3/users/f36035b9604948bf8e5c61d1c125cfbb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:16.188151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:16.277027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:16.285291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d7e0a-a035-4fc8-bead-80a87179e98a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:16.318046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ceb7a-82fd-4fa5-b463-74da0acb70cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:16.318726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ceb7a-82fd-4fa5-b463-74da0acb70cc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:16.319027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ceb7a-82fd-4fa5-b463-74da0acb70cc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:16.319292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ceb7a-82fd-4fa5-b463-74da0acb70cc None admin] PATH_INFO: `/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:16.320505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ceb7a-82fd-4fa5-b463-74da0acb70cc None admin] RBAC: Authorizing `identity:get_project(project_id=67fdc3ebe0024d54829c17e6da4ceb24)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:16.321944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ceb7a-82fd-4fa5-b463-74da0acb70cc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:16.323593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 573/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:16 2022] GET /identity/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:16.329532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:16.346178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:16.427643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178bdfa-9c5c-4001-ba82-1e049cedca21 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:16.460164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b81d-4925-42c8-bf81-20a17a9050e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:16.460793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b81d-4925-42c8-bf81-20a17a9050e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:16.461069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b81d-4925-42c8-bf81-20a17a9050e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:16.461337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b81d-4925-42c8-bf81-20a17a9050e9 None admin] PATH_INFO: `/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24/users/f36035b9604948bf8e5c61d1c125cfbb/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:16.463215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b81d-4925-42c8-bf81-20a17a9050e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=67fdc3ebe0024d54829c17e6da4ceb24, user_id=f36035b9604948bf8e5c61d1c125cfbb, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:16.464635 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b81d-4925-42c8-bf81-20a17a9050e9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:16.546202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 580/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:16:16 2022] PUT /identity/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24/users/f36035b9604948bf8e5c61d1c125cfbb/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:16:18.440077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bf5fa-e48a-4735-80aa-ab78e3b93e66 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:18.440635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bf5fa-e48a-4735-80aa-ab78e3b93e66 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:18.440635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bf5fa-e48a-4735-80aa-ab78e3b93e66 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:18.441969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 574/1154] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:18.447526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:18.447895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:18.448082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:18.466515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:18.467738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:18.532693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 581/1155] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:18.806240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bf5fa-e48a-4735-80aa-ab78e3b93e66 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:18.904710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bf5fa-e48a-4735-80aa-ab78e3b93e66 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:18.937454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:18.938069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:18.938363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:18.938639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:18.948707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:16:18.953381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:18.963690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:16:18.964637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 575/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:18.965353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:18.965353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:19.058130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700238-e2ce-4798-910a-7b23b50dd941 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:19.091995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:19.092700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:19.092979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:19.093249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:19.093907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:19.094207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:19.095604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:19.104192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 582/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:16:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:19.110032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:19.131628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:19.205702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52e172-0aaa-4494-9444-d13c1b40355b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:19.238741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:19.239361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:19.239641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:19.239912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] PATH_INFO: `/v3/users/f36035b9604948bf8e5c61d1c125cfbb` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:19.241034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] RBAC: Authorizing `identity:get_user(user_id=f36035b9604948bf8e5c61d1c125cfbb)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:19.242493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:19.244068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 576/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:19 2022] GET /identity/v3/users/f36035b9604948bf8e5c61d1c125cfbb => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:19.249853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:19.313019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:19.346240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad8884-4b86-44ce-99e4-a46aa4f67370 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:19.378875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:19.379464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:19.379741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:19.380006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:19.444905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:19.446370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:19.506025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:19.540749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 583/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:16:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:16:19.546620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:19.632978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:19.644268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fca02-9726-4741-8b38-dac9a4d2cc2b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:19.677492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf8584-f030-4020-a1a6-f51017b4a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:19.678935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf8584-f030-4020-a1a6-f51017b4a5ef None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:19.679373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf8584-f030-4020-a1a6-f51017b4a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:19.679827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf8584-f030-4020-a1a6-f51017b4a5ef None admin] PATH_INFO: `/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:19.681619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf8584-f030-4020-a1a6-f51017b4a5ef None admin] RBAC: Authorizing `identity:get_project(project_id=67fdc3ebe0024d54829c17e6da4ceb24)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:19.684182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf8584-f030-4020-a1a6-f51017b4a5ef None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:19.687055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 577/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:19 2022] GET /identity/v3/projects/67fdc3ebe0024d54829c17e6da4ceb24 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:19.693178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:19.789978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04b7cb-0fd4-4f86-b51e-07a45ce37be1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:19.825010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:19.825608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:19.825887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:19.826154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:19.826911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=67fdc3ebe0024d54829c17e6da4ceb24, user.id=f36035b9604948bf8e5c61d1c125cfbb) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:19.827216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:19.828636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1b2c1-90bb-44e0-b874-ccc0ddf0334c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:19.835060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 584/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:16:19 2022] GET /identity/v3/role_assignments?user.id=f36035b9604948bf8e5c61d1c125cfbb&scope.project.id=67fdc3ebe0024d54829c17e6da4ceb24&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:21.745114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52084134-0c18-463c-bf08-5fa80fe21847 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:21.745672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52084134-0c18-463c-bf08-5fa80fe21847 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:21.745672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52084134-0c18-463c-bf08-5fa80fe21847 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:21.747040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 578/1162] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:21.752152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:21.752559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:21.752709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:21.771413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:21.773182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:21.796054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:21.838236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 585/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:22.102924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52084134-0c18-463c-bf08-5fa80fe21847 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:22.200394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52084134-0c18-463c-bf08-5fa80fe21847 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:22.232776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:22.233367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:22.233644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:22.233910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:22.257634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:22.300112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:22.301854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:22.397508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 579/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:16:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:16:22.403447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:22.499833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e9bdd-981b-4e97-9a46-7a1760462994 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:22.533520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:22.534138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:22.534444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:22.534714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:22.541259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:22.542780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0a011f5e-4ccf-4724-ac0d-ceb1e87e5bb3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:22.553938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:22.560035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:16:22.561138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 586/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:16:22 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:22.562165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:22.616329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:22.659193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007cdaf-c9ec-4516-8d2b-d44d61d82560 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:22.692653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:22.693247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:22.694078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:22.694422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:22.695235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:22.695535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:22.696893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a008-9b91-48c8-9433-bc31d00b837a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:22.703831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 580/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Sep 3 01:16:22 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:16:24.561009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e6a4d-9cab-4373-9da5-ace943f0968e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:24.561538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e6a4d-9cab-4373-9da5-ace943f0968e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:24.561538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e6a4d-9cab-4373-9da5-ace943f0968e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:24.562916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 587/1167] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:24.567950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c472a-d2a9-48a6-b453-d3fdcbd0d74f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:24.568321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c472a-d2a9-48a6-b453-d3fdcbd0d74f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:24.569026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c472a-d2a9-48a6-b453-d3fdcbd0d74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:24.586770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7b8c472a-d2a9-48a6-b453-d3fdcbd0d74f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:24.587943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7b8c472a-d2a9-48a6-b453-d3fdcbd0d74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:24.653234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 581/1168] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:24.920896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e6a4d-9cab-4373-9da5-ace943f0968e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:24.943455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:25.021348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e6a4d-9cab-4373-9da5-ace943f0968e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:25.054467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97344234-d23b-4213-8077-126c5453de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:25.055050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97344234-d23b-4213-8077-126c5453de4e None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:25.055328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97344234-d23b-4213-8077-126c5453de4e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:25.055595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97344234-d23b-4213-8077-126c5453de4e None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:25.056265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97344234-d23b-4213-8077-126c5453de4e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:25.057627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97344234-d23b-4213-8077-126c5453de4e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:25.129407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 588/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Sat Sep 3 01:16:24 2022] POST /identity/v3/domains => generated 267 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:16:27.047745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2b1d9-59fa-4a56-b6ee-d8ca92adc870 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:27.048264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2b1d9-59fa-4a56-b6ee-d8ca92adc870 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:27.048350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2b1d9-59fa-4a56-b6ee-d8ca92adc870 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:27.049645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 582/1170] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:27.055068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:27.055440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:27.055621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:27.074408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:27.075607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:27.144126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 589/1171] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:27.414254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2b1d9-59fa-4a56-b6ee-d8ca92adc870 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:27.463973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:27.512353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2b1d9-59fa-4a56-b6ee-d8ca92adc870 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:27.544498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4e095-b4ba-430f-91b4-81075cf31481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:27.545116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4e095-b4ba-430f-91b4-81075cf31481 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:27.545397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4e095-b4ba-430f-91b4-81075cf31481 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:27.545667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4e095-b4ba-430f-91b4-81075cf31481 None admin] PATH_INFO: `/v3/domains/4bfe111caba04a38a48070b270e420d7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:27.556529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4e095-b4ba-430f-91b4-81075cf31481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4bfe111caba04a38a48070b270e420d7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:27.557983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4e095-b4ba-430f-91b4-81075cf31481 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:27.559576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 583/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Sat Sep 3 01:16:27 2022] GET /identity/v3/domains/4bfe111caba04a38a48070b270e420d7 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:27.565306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:27.577285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:27.665619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10027b-b396-4e5b-97cb-2178cb4cfd34 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:27.699304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5a5ae-ab3f-4328-880b-aaffa4231802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:27.699906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5a5ae-ab3f-4328-880b-aaffa4231802 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:27.700187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5a5ae-ab3f-4328-880b-aaffa4231802 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:27.700455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5a5ae-ab3f-4328-880b-aaffa4231802 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:27.701108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5a5ae-ab3f-4328-880b-aaffa4231802 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:27.702640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5a5ae-ab3f-4328-880b-aaffa4231802 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:27.785911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 590/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:16:27 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:16:29.724841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258129-8bc1-460d-a9bc-2da78aad2fee None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:29.725374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258129-8bc1-460d-a9bc-2da78aad2fee None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:29.725374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86258129-8bc1-460d-a9bc-2da78aad2fee None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:29.726913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 584/1174] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:29.732797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:29.733225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:29.733351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:29.751258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:29.752445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:29.807931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:29.817942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 591/1175] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:30.091862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258129-8bc1-460d-a9bc-2da78aad2fee None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:30.190353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86258129-8bc1-460d-a9bc-2da78aad2fee None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:30.201025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:30.225917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3683f1-1a41-4dcc-a2cb-a3532a00259a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:30.227446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3683f1-1a41-4dcc-a2cb-a3532a00259a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:30.227446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3683f1-1a41-4dcc-a2cb-a3532a00259a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:30.227446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3683f1-1a41-4dcc-a2cb-a3532a00259a None admin] PATH_INFO: `/v3/domains/4bfe111caba04a38a48070b270e420d7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:30.228272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3683f1-1a41-4dcc-a2cb-a3532a00259a None admin] RBAC: Authorizing `identity:get_domain(domain_id=4bfe111caba04a38a48070b270e420d7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:30.229824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3683f1-1a41-4dcc-a2cb-a3532a00259a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:30.231489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 585/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Sat Sep 3 01:16:30 2022] GET /identity/v3/domains/4bfe111caba04a38a48070b270e420d7 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:30.237496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:30.335504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b48ab6-1302-480a-8070-59cb30c4f2c8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:30.368739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ebd69-eea6-4c5e-83ef-bbe2672a52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:30.369266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ebd69-eea6-4c5e-83ef-bbe2672a52e7 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:30.369542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ebd69-eea6-4c5e-83ef-bbe2672a52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:30.369808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ebd69-eea6-4c5e-83ef-bbe2672a52e7 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:30.370487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ebd69-eea6-4c5e-83ef-bbe2672a52e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:30.371910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ebd69-eea6-4c5e-83ef-bbe2672a52e7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:30.441986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 592/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:16:30 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:16:32.379545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb0802-8c86-416f-8333-79c18527d54c None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:32.380075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb0802-8c86-416f-8333-79c18527d54c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:32.380075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb0802-8c86-416f-8333-79c18527d54c None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:32.381979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 586/1178] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:32.387356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:32.387764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:32.387908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:32.406645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:32.407842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:32.451156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:32.474141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 593/1179] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:32.747234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb0802-8c86-416f-8333-79c18527d54c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:32.847724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb0802-8c86-416f-8333-79c18527d54c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:32.881371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:32.881993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:32.882273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:32.882667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:32.892786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:16:32.898764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:32.899941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:16:32.901050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 587/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:32.906021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:33.000825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb7699-5b26-4955-b278-acc27ad66837 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:33.095898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:33.096501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:33.096781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:33.097052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:33.097710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:33.098011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:33.099552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:33.107351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 594/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:33.113059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:33.152231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:33.211016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc53dd-0c98-4e12-98bd-7c39dff2a2a2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:33.243113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:33.243735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:33.244017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:33.244290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] PATH_INFO: `/v3/users/886339de97484d2ea6569e797ff9ac5e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:33.258526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] RBAC: Authorizing `identity:get_user(user_id=886339de97484d2ea6569e797ff9ac5e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:33.259911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:33.261487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 588/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:33 2022] GET /identity/v3/users/886339de97484d2ea6569e797ff9ac5e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:33.267573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:33.309274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:33.365880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb98e25-f203-424b-88f8-4f8fcd3d6881 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:33.398932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:33.399515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:33.399788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:33.400055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:33.465181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:33.466705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:33.504922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:33.561099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 595/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:16:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:16:33.566948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:33.657015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:33.663305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeddef-8a88-4b3f-ab23-df423f859c2e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:33.696247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df8717-5486-4e35-8fd5-61c12193773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:33.696866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df8717-5486-4e35-8fd5-61c12193773f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:33.697144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df8717-5486-4e35-8fd5-61c12193773f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:33.697414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df8717-5486-4e35-8fd5-61c12193773f None admin] PATH_INFO: `/v3/projects/31a9d8f9c0934e9387fca701ef2da176` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:33.698535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df8717-5486-4e35-8fd5-61c12193773f None admin] RBAC: Authorizing `identity:get_project(project_id=31a9d8f9c0934e9387fca701ef2da176)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:33.699954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df8717-5486-4e35-8fd5-61c12193773f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:33.701539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 589/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:33 2022] GET /identity/v3/projects/31a9d8f9c0934e9387fca701ef2da176 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:33.707582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:33.805123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b29be5-df6b-43c5-b52b-6fa065fad787 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:33.838298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:33.838865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:33.839141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:33.839408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:33.840129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=31a9d8f9c0934e9387fca701ef2da176, user.id=886339de97484d2ea6569e797ff9ac5e) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:33.840440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:33.841854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da423d9a-5ac3-4791-a774-a7044680f3db None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:33.847106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 596/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:16:33 2022] GET /identity/v3/role_assignments?user.id=886339de97484d2ea6569e797ff9ac5e&scope.project.id=31a9d8f9c0934e9387fca701ef2da176&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:35.713431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6525cc-367a-4228-9e3b-d8825f0456a1 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:35.713991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6525cc-367a-4228-9e3b-d8825f0456a1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:35.713991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6525cc-367a-4228-9e3b-d8825f0456a1 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:35.715329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 590/1186] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:35.720377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:35.720775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:35.720920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:35.728739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:35.739301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:35.743048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:35.806264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 597/1187] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:16:36.073004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6525cc-367a-4228-9e3b-d8825f0456a1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:36.170808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6525cc-367a-4228-9e3b-d8825f0456a1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:36.203272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:36.203932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:36.204209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:36.204478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:36.214795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:16:36.219482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:36.220619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:16:36.221716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 591/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:36.226736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:36.323282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbdf36-c9fa-4713-a0ce-ae05a71de05a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:36.359041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:36.359651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:36.359935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:36.360206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:36.360869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:36.361173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:36.362572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:36.363906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:36.374349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 598/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:36.380419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:36.406084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:36.477590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be0770-c2c6-40bb-af85-a82e00124fb7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:36.510485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:36.511101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:36.511379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:36.511649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] PATH_INFO: `/v3/users/886339de97484d2ea6569e797ff9ac5e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:36.512741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] RBAC: Authorizing `identity:get_user(user_id=886339de97484d2ea6569e797ff9ac5e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:36.514167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:36.515737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 592/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:36 2022] GET /identity/v3/users/886339de97484d2ea6569e797ff9ac5e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:36.522016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:36.619035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b300e7-01e5-4e5a-8532-e838b0998ddb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:36.651370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d9090-78b2-4c4c-a649-5e9abfbf3bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:36.651999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d9090-78b2-4c4c-a649-5e9abfbf3bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:36.652273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d9090-78b2-4c4c-a649-5e9abfbf3bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:36.652541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d9090-78b2-4c4c-a649-5e9abfbf3bc3 None admin] PATH_INFO: `/v3/projects/31a9d8f9c0934e9387fca701ef2da176` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:36.653630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d9090-78b2-4c4c-a649-5e9abfbf3bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=31a9d8f9c0934e9387fca701ef2da176)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:36.655214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d9090-78b2-4c4c-a649-5e9abfbf3bc3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:36.656809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 599/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:36 2022] GET /identity/v3/projects/31a9d8f9c0934e9387fca701ef2da176 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:36.663293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:36.752970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:36.762011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccfc0a-1ef8-49ed-a02c-be9cd205facf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:36.795809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3257a-7d73-4253-8716-5c31d5f089cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:36.796456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3257a-7d73-4253-8716-5c31d5f089cd None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:36.796741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3257a-7d73-4253-8716-5c31d5f089cd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:36.797018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3257a-7d73-4253-8716-5c31d5f089cd None admin] PATH_INFO: `/v3/projects/31a9d8f9c0934e9387fca701ef2da176/users/886339de97484d2ea6569e797ff9ac5e/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:36.798970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3257a-7d73-4253-8716-5c31d5f089cd None admin] RBAC: Authorizing `identity:create_grant(project_id=31a9d8f9c0934e9387fca701ef2da176, user_id=886339de97484d2ea6569e797ff9ac5e, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:36.800445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3257a-7d73-4253-8716-5c31d5f089cd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:36.853781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 593/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:16:36 2022] PUT /identity/v3/projects/31a9d8f9c0934e9387fca701ef2da176/users/886339de97484d2ea6569e797ff9ac5e/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:16:38.732111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f682d-d640-4bcf-aa5b-ce81d425f718 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:38.732641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f682d-d640-4bcf-aa5b-ce81d425f718 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:38.732641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f682d-d640-4bcf-aa5b-ce81d425f718 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:38.734003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 600/1193] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:16:38.739240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:38.740355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:38.740355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:38.758713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:16:38.759888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:16:38.824178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 594/1194] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:16:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:16:39.092016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f682d-d640-4bcf-aa5b-ce81d425f718 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:39.154095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:39.191556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f682d-d640-4bcf-aa5b-ce81d425f718 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:39.223859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:39.224991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:39.225264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:39.225525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:39.235208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:16:39.240387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:16:39.241527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:16:39.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 601/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:16:39.247514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:39.327539 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:39.345011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb19f6-193f-4afc-8e0c-09b602b70812 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:39.378184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:39.379039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:39.379483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:39.380128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:39.381052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:39.381354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:39.382770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:39.390715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 595/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:39.397303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:39.494042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d347ffd-0b7a-4cb8-9f7e-71bf0cb2d289 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:39.516183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:39.527332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:39.527948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:39.528227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:39.528495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] PATH_INFO: `/v3/users/886339de97484d2ea6569e797ff9ac5e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:39.529598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] RBAC: Authorizing `identity:get_user(user_id=886339de97484d2ea6569e797ff9ac5e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:39.531050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:39.532639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 602/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Sep 3 01:16:39 2022] GET /identity/v3/users/886339de97484d2ea6569e797ff9ac5e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:39.538982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:39.635585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db25fd-8be0-4a13-9676-76a650dd7609 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:39.668075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:39.668669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:39.668945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:39.669219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:39.682790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:39.733724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:39.735199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:39.830446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 596/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:16:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:16:39.836364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:39.933020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc68d52-82f4-4768-802e-65d51164dbfe None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:39.965055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a9b3-9fc9-4d12-b887-2461bb6dfb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:39.965665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a9b3-9fc9-4d12-b887-2461bb6dfb58 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:39.965946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a9b3-9fc9-4d12-b887-2461bb6dfb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:39.966219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a9b3-9fc9-4d12-b887-2461bb6dfb58 None admin] PATH_INFO: `/v3/projects/31a9d8f9c0934e9387fca701ef2da176` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:39.967348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442a9b3-9fc9-4d12-b887-2461bb6dfb58 None admin] RBAC: Authorizing `identity:get_project(project_id=31a9d8f9c0934e9387fca701ef2da176)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:39.968781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442a9b3-9fc9-4d12-b887-2461bb6dfb58 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:39.970411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 603/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Sep 3 01:16:39 2022] GET /identity/v3/projects/31a9d8f9c0934e9387fca701ef2da176 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:39.976977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:16:40.021275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:40.073969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c302955-b28a-49e1-8f50-6b0093a13778 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:16:40.107862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:16:40.108459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:16:40.108741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:16:40.109012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:16:40.109738 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=31a9d8f9c0934e9387fca701ef2da176, user.id=886339de97484d2ea6569e797ff9ac5e) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:16:40.110048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:16:40.111516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f139b-7464-4d3a-a3d8-2a33dae66e4a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:16:40.117114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 597/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:16:39 2022] GET /identity/v3/role_assignments?user.id=886339de97484d2ea6569e797ff9ac5e&scope.project.id=31a9d8f9c0934e9387fca701ef2da176&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:16:49.961103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=70 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:16:50.115238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=46 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:50.745343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4ad5-8fd5-460a-9eeb-436ff491e664 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:50.745977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4ad5-8fd5-460a-9eeb-436ff491e664 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:50.745977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4ad5-8fd5-460a-9eeb-436ff491e664 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:50.747265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 604/1201] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:17:50.752409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:50.754471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:50.754471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:50.771826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:17:50.773016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:17:50.839512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 598/1202] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:17:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:17:51.109128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc4ad5-8fd5-460a-9eeb-436ff491e664 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:51.210536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc4ad5-8fd5-460a-9eeb-436ff491e664 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:51.244041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:51.244669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:51.244946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:51.245212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:51.254921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:51.257126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:17:51.271463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:51.274625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:17:51.275620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:17:51.275620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:51.275620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:17:51.275620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 605/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:51.279707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:51.379549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d151c-3194-4e6c-970d-0c5e2921a398 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:51.408754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:51.415009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:51.415612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:51.415894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:51.416167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:51.416799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:51.417103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:51.418515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28fe58-2396-47e7-8b49-cbdcca206422 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:51.429462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 599/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:51.435452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:51.488058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:51.535126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc904a-5c02-44b8-95fd-b9b91a92e1f0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:51.569310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a5869-d83f-4f27-97e8-dc25b2c50724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:51.569901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5869-d83f-4f27-97e8-dc25b2c50724 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:51.577760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5869-d83f-4f27-97e8-dc25b2c50724 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:51.577760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a5869-d83f-4f27-97e8-dc25b2c50724 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:51.577760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a5869-d83f-4f27-97e8-dc25b2c50724 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:51.577760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a5869-d83f-4f27-97e8-dc25b2c50724 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:51.686890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 606/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Sep 3 01:17:51 2022] POST /identity/v3/users => generated 251 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:17:53.650970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74914c94-98c7-4257-a540-2d60bb781b46 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:53.651535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74914c94-98c7-4257-a540-2d60bb781b46 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:53.651535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74914c94-98c7-4257-a540-2d60bb781b46 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:53.652902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 600/1206] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:17:53.657972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:53.658375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:53.658559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:53.678475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:17:53.679730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:17:53.752239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 607/1207] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:17:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:17:54.027569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74914c94-98c7-4257-a540-2d60bb781b46 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:54.128799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74914c94-98c7-4257-a540-2d60bb781b46 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:54.160973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:54.161594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:54.162428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:54.162723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:54.172884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:17:54.178177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:54.179397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:54.180452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 601/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:54.187687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:54.228012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:54.286131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270d5d7-3436-4969-a68e-ea9f658a86bb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:54.322098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:54.322745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:54.323027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:54.323299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:54.323966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:54.324268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:54.325645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:54.333514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 608/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:54.339899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:54.389401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:54.438299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ade5f7-85a1-4498-ba44-72180ab1dd0f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:54.472553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:54.473195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:54.473458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:54.473731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:54.484323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:54.489552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:17:54.499940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:54.501073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:17:54.502363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:17:54.502363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:54.502363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:54.502363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 602/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:54.507733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:54.604364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:54.607233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f961a19-3573-48b5-9dc8-8eb331cf6afb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:54.645767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:54.647329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:54.647696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:54.647976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:54.648622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:54.648926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:54.650377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:54.661707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 609/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:54.667449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:54.765642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00cfb1-c5ca-44aa-bc1e-dec9c81c3f6f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:54.767183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:54.799518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:54.800156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:54.800437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:54.800729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:54.808954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:54.810471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:17:54.821949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:54.823094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:54.824271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 603/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:54.828966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:54.929900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435b70e-289e-4a2d-8296-887b27fb8765 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:54.963297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:54.963905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:54.964191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:54.964467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:54.965195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:54.965502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:54.966972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:54.984605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 610/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:54.990567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:55.090452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f93dd-b02e-4ef9-9371-6bd754a03acf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:55.124594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:55.125193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:55.125471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:55.125739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:55.156189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:55.192128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:55.193569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:55.290385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 604/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:17:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:17:55.296228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:55.314928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:55.398763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debabe-7fc8-4726-a17b-d24b956e6c11 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:55.433751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:55.434398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:55.434684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:55.434959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:55.445672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:55.447092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:17:55.462680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:55.463874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:17:55.464906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:17:55.464906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:55.464906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:17:55.464906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 611/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:55.469536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:55.531188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:55.567501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e66848-ee81-472b-bc29-3a9fb39c3bb7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:55.600663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:55.601268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:55.601541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:55.601811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:55.602618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:55.602924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:55.604600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:55.615313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 605/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:55.621161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:55.690159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:55.721018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01978428-30ac-48ec-8b6e-f789f1e8cfc0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:55.753060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:55.753793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:55.754013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:55.754277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:55.761370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:55.762843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:17:55.774276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:55.775376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:17:55.776708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 612/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:17:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:55.781293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:55.882755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04809a-e635-49d9-967a-c482c1e960bb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:55.906122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:55.915718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:55.916325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:55.916606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:55.916878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:55.917562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:55.917870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:55.919403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475e073-2186-4f21-a8f2-002a0f45e307 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:55.931926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 606/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:17:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:55.938026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:56.038107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b251c-ff13-4827-aec9-93265002dc00 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:56.066575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:56.072818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bc492-749c-4284-aca2-49899255833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:56.073419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc492-749c-4284-aca2-49899255833b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:56.073701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc492-749c-4284-aca2-49899255833b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:56.073971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bc492-749c-4284-aca2-49899255833b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:56.074720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bc492-749c-4284-aca2-49899255833b None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=54b10e8ec2e244898f250cb987b6c7c8) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:56.075028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bc492-749c-4284-aca2-49899255833b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:56.076443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bc492-749c-4284-aca2-49899255833b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:56.082654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 613/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:17:55 2022] GET /identity/v3/role_assignments?user.id=54b10e8ec2e244898f250cb987b6c7c8&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:57.993390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d4569-e377-4813-af24-5fbd088a7dc2 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:57.993917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d4569-e377-4813-af24-5fbd088a7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:57.993917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d4569-e377-4813-af24-5fbd088a7dc2 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:57.995329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 607/1220] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:17:58.000624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:58.001046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:58.001209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:58.019935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:17:58.021388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:17:58.089446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 614/1221] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:17:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:17:58.370531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d4569-e377-4813-af24-5fbd088a7dc2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:58.475506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d4569-e377-4813-af24-5fbd088a7dc2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:58.509591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:58.510214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:58.510605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:58.510882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:58.521033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:17:58.526160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:58.527318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:17:58.528362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:58.530492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 608/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:17:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:58.533311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:58.632539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bcf57-03b1-43f8-941a-52d5fcfc91ad None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:58.661001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:58.672116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:58.676155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:58.676155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:58.676155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:58.676155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:58.676155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:58.676155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:58.684137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 615/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:17:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:17:58.689974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:58.736026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:58.792572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a4fbd-cb4f-4dbc-92e6-c503fc819719 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:58.829511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:58.830146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:58.830463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:58.830741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:58.840633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:58.842083 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:17:58.856726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:58.857929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:17:58.859181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:17:58.859181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:58.859181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:58.859181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 609/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:58.863922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:58.964593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84662a-328a-443a-87aa-f1349a0a978a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:58.998219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:58.998859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:58.999140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:58.999470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.000108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:59.000421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.001799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.015226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 616/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:59.021359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:59.114809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:59.121896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddda1a-7997-4427-9a8a-560f7429ab4c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:59.154680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:59.155306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:59.155595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:59.155867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.163773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.165265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:17:59.176128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:59.177203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:59.179052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 610/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:17:59.184023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:59.230781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:59.282459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e882c-0cee-4d67-a50b-9e222f57d19b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:59.316449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:59.317067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:59.317341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:59.317604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.318319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:59.318769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.320206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.337691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 617/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Sep 3 01:17:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:59.343541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:59.444573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c20a2d-8bbb-4524-b35a-f1a485079a98 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:59.477889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:59.478607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:59.478886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:59.479156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.489880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.491988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:17:59.505599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:59.508696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:17:59.509823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:17:59.509823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:17:59.509823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:59.509823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 611/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:59.513401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:59.612747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19e306-5b07-41ef-a467-28dc915711d9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:59.615067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:59.645953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:59.647362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:59.647645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:59.647905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.648557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:59.648856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.650227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.661698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 618/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:59.668060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:59.677385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:17:59.766296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f414dd9-fe1a-4ce0-8c51-7d96c5493a45 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:59.803017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:59.804010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:59.804406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:59.804679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.811690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.813137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:17:59.825100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:17:59.826209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:17:59.827254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 612/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:17:59.832327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:17:59.936157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa317e-16cb-4867-ac7c-970e6c55a315 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:17:59.970896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:17:59.971538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:17:59.971818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:17:59.972084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:17:59.972768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:17:59.973069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:17:59.974521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81105e4e-22dd-4131-b4d3-de6ce585b56f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:17:59.986280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 619/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:17:59.992588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:00.059493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:00.097733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f7b20-1461-4589-a62b-3c70344561eb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:00.132077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f33bf-b08b-4ff6-bb33-e16ded4f577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:00.132755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f33bf-b08b-4ff6-bb33-e16ded4f577b None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:00.133036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f33bf-b08b-4ff6-bb33-e16ded4f577b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:00.133305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f33bf-b08b-4ff6-bb33-e16ded4f577b None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/54b10e8ec2e244898f250cb987b6c7c8/roles/f49b53788e2946ffb5cd76dc2ddbd589` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:00.147896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f33bf-b08b-4ff6-bb33-e16ded4f577b None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=54b10e8ec2e244898f250cb987b6c7c8, role_id=f49b53788e2946ffb5cd76dc2ddbd589)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:00.149328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f33bf-b08b-4ff6-bb33-e16ded4f577b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:00.229418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 613/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:17:59 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/54b10e8ec2e244898f250cb987b6c7c8/roles/f49b53788e2946ffb5cd76dc2ddbd589 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:18:02.223828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dea05-c0c0-4188-bdb3-364cc89cfef8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:02.224365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dea05-c0c0-4188-bdb3-364cc89cfef8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:02.224365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dea05-c0c0-4188-bdb3-364cc89cfef8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:02.226320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 620/1233] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:02.231365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:02.231780 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:02.231902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:02.250224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:02.251473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:02.316770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 614/1234] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:02.587708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dea05-c0c0-4188-bdb3-364cc89cfef8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:02.613363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:02.695094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dea05-c0c0-4188-bdb3-364cc89cfef8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:02.729692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:02.730380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:02.733007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:02.733007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:02.742306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:18:02.747271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:02.748490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:02.749423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 621/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Sep 3 01:18:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:02.754685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:02.801033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:02.853457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f1a62-5a34-489d-84ac-fae5c533382d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:02.887292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:02.887903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:02.888187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:02.888462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:02.889131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:02.889444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:02.890863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:02.899523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 615/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Sep 3 01:18:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:02.905250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:02.989556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:03.005029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645aa2e8-e6f1-4a75-a790-f66a5d5776dc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:03.039845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:03.040462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:03.040736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:03.041001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:03.050642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:03.052073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:03.066305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:03.069836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:03.070998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:03.070998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:03.070998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:03.070998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 622/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:03.074247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:03.172451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93864e3-14ca-40ec-be8b-0b4883b94fea None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:03.181541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:03.205610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:03.206216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:03.206531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:03.206803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:03.207443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:03.207745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:03.209124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:03.220406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 616/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:03.226474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:03.327033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe9b67-0811-4ed6-97f6-2061a74c9d71 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:03.359721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:03.360345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:03.360625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:03.360894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] PATH_INFO: `/v3/users/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:03.363518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:03.368896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:03.370353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:18:03.381899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:03.383009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:03.384006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 623/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:03.389399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:03.488341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee44266-f906-423c-8580-bb70f768aa7a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:03.523497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:03.524101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:03.524383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:03.524653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:03.525404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:03.525706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:03.527171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:03.544381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 617/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Sep 3 01:18:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:18:03.550324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:03.651926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0ec22-3f8a-4edf-ae4f-012b6a03e5de None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:03.684606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:03.685216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:03.685570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:03.685836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:03.746284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:03.752759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:03.754218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:03.853933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 624/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:18:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:18:03.859918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:03.887778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:03.960996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055341fa-6396-4f2b-a1e2-1b0872d60c8b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:03.995732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:04.001287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:04.001287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:04.001287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:04.007598 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:04.009054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:04.023816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:04.025187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:04.026076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:04.026076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:04.026076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:04.026076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 618/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:04.030795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:04.122801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:04.128624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905a943-cff6-4b91-94a8-23f74196d4bd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:04.163092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:04.163688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:04.163965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:04.164232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:04.164917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:04.165221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:04.166615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:04.177511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 625/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:04.183733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:04.261901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:04.283359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810e9b-2947-44cb-8fcf-269fc36d977c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:04.316927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:04.317583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:04.317864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:04.318132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:04.325314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:04.326848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:18:04.338113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:04.339118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:04.340065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 619/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:18:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:04.345138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:04.447414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fd1b1-1130-4c37-a3f4-15ffae3fc4be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:04.480123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:04.480727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:04.481017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:04.481288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:04.482011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:04.482281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:04.485790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450b949-907a-4553-b0f8-2002ca382c26 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:04.496223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 626/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:04.502487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:04.603440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d22130-32b2-4801-8fcb-a2038dc7642f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:04.636825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:04.637427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:04.637708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:04.637980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:04.638793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] RBAC: Adding query filter params (role.id=f49b53788e2946ffb5cd76dc2ddbd589, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=54b10e8ec2e244898f250cb987b6c7c8) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:04.639102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:04.640560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0aa4e6-6c06-4f4b-8cd1-c3b9e9071d70 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:04.646438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 620/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:18:04 2022] GET /identity/v3/role_assignments?user.id=54b10e8ec2e244898f250cb987b6c7c8&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=f49b53788e2946ffb5cd76dc2ddbd589 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:06.617055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16221f-376f-473b-b6ff-fbcbf845069a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:06.617623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16221f-376f-473b-b6ff-fbcbf845069a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:06.617623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16221f-376f-473b-b6ff-fbcbf845069a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:06.619017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 627/1247] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:06.624261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:06.624684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:06.624849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:06.631304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:06.643374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:06.644570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:06.715777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 621/1248] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:06.995957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16221f-376f-473b-b6ff-fbcbf845069a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:07.014879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:07.099991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16221f-376f-473b-b6ff-fbcbf845069a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:07.134185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:07.134848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:07.135128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:07.135397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:07.145375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:18:07.150509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:07.151554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:07.152459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 628/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:07.157608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:07.256637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b84bd4-eb57-4b17-a6d7-5455d3c0d28d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:07.285663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:07.290608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:07.291268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:07.291549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:07.291819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:07.292482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:07.292784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:07.294158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:07.301967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 622/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:18:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:07.308446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:07.386604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:07.408992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cf546-788a-4715-b3df-cd40544e442a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:07.442641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:07.443302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:07.443581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:07.443851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:07.453624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:07.455093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:07.470242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:07.472014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:07.473141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:07.473141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:07.473141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:07.473141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 629/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:07.478076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:07.579912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894ba30-b562-4255-86af-fce17b805ad9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:07.613045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:07.613697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:07.613980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:07.614251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:07.614919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:07.615251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:07.616623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:07.627979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 623/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:07.634320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:07.733552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cec0a3-be44-4d40-a838-2ab316d6f7ae None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:07.760483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:07.768253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:07.768883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:07.769165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:07.769440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:07.778165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:07.779661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:18:07.790806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:07.791930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:07.793195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 630/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:07.797755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:07.854244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:07.897611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06feded-b070-4bd1-8084-5c86d6433d08 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:07.931806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:07.932414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:07.932792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:07.933117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:07.933904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:07.934256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:07.935983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:07.953678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 624/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Sep 3 01:18:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:07.959711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:08.061478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2629b-4e4d-4b1b-9d78-7932fb4bcd45 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:08.095178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:08.095784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:08.096066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:08.096338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:08.151299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:08.163889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:08.165394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:08.262353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 631/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:18:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:18:08.268225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:08.365899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b600bb-e17c-480a-915f-322e43b3f6f9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:08.378594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:08.400312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:08.400935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:08.401213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:08.401535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:08.410998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:08.412515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:08.426352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:08.427474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:08.428907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:08.428907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:08.428907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:08.428907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 625/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:08.433694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:08.520863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:08.533752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174431-d456-43c4-8161-6f02e51a8fcb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:08.566941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:08.567539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:08.567819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:08.568747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:08.569378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:08.569679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:08.571068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:08.581877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 632/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:08.587631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:08.684863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ae84-f9f0-401d-bf7d-9304a8bc48a8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:08.726141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:08.729992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:08.729992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:08.729992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:08.735815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:08.737338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:18:08.748997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:08.750097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:08.751266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 626/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:18:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:08.758724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:08.858500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37787-886d-4e2c-a113-191a7e3170ad None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:08.891329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:08.891910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:08.892194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:08.892470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:08.893165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:08.893563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:08.894951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f0824-0783-49e0-8718-f5517283decd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:08.907319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 633/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:18:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:08.914310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:08.941381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:09.013769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184dcf6-203e-491a-ac7d-117eb607bcf0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:09.047586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:09.048190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:09.048470 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:09.048740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:09.049459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=54b10e8ec2e244898f250cb987b6c7c8) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:09.049763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:09.051273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99b17f-c519-45cc-9440-d2e568700b4b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:09.057620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 627/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:18:08 2022] GET /identity/v3/role_assignments?user.id=54b10e8ec2e244898f250cb987b6c7c8&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:10.938196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4303a-a9ec-43c2-a8c4-431a3341cdce None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:10.938787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4303a-a9ec-43c2-a8c4-431a3341cdce None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:10.938787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4303a-a9ec-43c2-a8c4-431a3341cdce None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:10.940135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 634/1261] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:10.945354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:10.945745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:10.945904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:10.964727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:10.965927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:11.032032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 628/1262] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:11.300135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4303a-a9ec-43c2-a8c4-431a3341cdce None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:11.311703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:11.402986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4303a-a9ec-43c2-a8c4-431a3341cdce None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:11.437868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:11.438520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:11.438818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:11.439072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:11.449177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:18:11.454008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:11.455013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:11.455853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 635/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:18:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:11.462813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:11.481246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:11.560865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3fd7d-e9a6-4c95-9728-4bccb8def2ea None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:11.595254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:11.595855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:11.596131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:11.596404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:11.597065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:11.597366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:11.598776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:11.607668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 629/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:18:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:11.613483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:11.680502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:11.713940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5594-deed-4378-8949-690ebf4f6f71 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:11.747186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:11.747803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:11.748198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:11.748470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:11.758750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:11.760180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:11.774688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:11.775899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:11.777122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:11.777122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:11.777122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:11.777122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 636/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:11.781593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:11.867794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:11.887313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830f9cc-6cf8-4a2b-a72d-258e6a3a9ba5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:11.920205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:11.920809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:11.921091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:11.921390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:11.922031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:11.922363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:11.923746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:11.935356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 630/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:11.941139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:12.042997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932377bf-017b-4e1f-b260-e24b2881c296 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:12.062784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:12.077023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:12.077659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:12.077940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:12.078213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:12.086398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:12.092703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:18:12.099062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:12.100427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:12.101453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 637/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:12.106113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:12.207328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc5398-6a91-448f-ad25-957a365e3e13 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:12.241035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:12.241629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:12.241905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:12.242174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:12.242922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:12.243222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:12.244652 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:12.253428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:12.262530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 631/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Sep 3 01:18:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:12.268272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:12.369431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafaed7-09be-442a-9736-cae6f9ec8eb6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:12.403362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:12.404040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:12.404315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:12.404587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:12.415655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:12.417196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:12.431091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:12.432770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:12.433976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:12.433976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:12.433976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:12.434667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 638/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:12.438167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:12.539435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920dc64-1010-452f-967f-a9323d2961a8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:12.573050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:12.573652 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:12.573930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:12.574201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:12.574862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:12.575163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:12.576565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:12.588245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 632/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:12.594087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:12.624812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:12.694129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99672f7-799c-4a19-8ae9-97c3f451502e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:12.734214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:12.734867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:12.735148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:12.735417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:12.742766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:12.744229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:18:12.756940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:12.758132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:12.759325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 639/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:18:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 03 01:18:12.764155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:12.822909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:12.863450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59894f4d-743f-41f9-874f-7485e946ef6f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:12.895398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:12.895997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:12.897226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:12.897494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:12.898177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:12.898507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:12.899933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5645def4-9e49-4258-ade8-086bfef202c8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:12.912526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 633/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:12.918598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:13.008694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:13.017027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ccc49-d9b3-4fe2-b36a-5ecab6c97609 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:13.049921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48ecf2-1870-4fea-a849-d230462bc2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:13.050592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48ecf2-1870-4fea-a849-d230462bc2ee None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:13.050955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48ecf2-1870-4fea-a849-d230462bc2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:13.051139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48ecf2-1870-4fea-a849-d230462bc2ee None admin] PATH_INFO: `/v3/projects/9f9529286db449018f980b53c7eb4e67/users/54b10e8ec2e244898f250cb987b6c7c8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:13.054287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48ecf2-1870-4fea-a849-d230462bc2ee None admin] RBAC: Authorizing `identity:create_grant(project_id=9f9529286db449018f980b53c7eb4e67, user_id=54b10e8ec2e244898f250cb987b6c7c8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:13.055050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48ecf2-1870-4fea-a849-d230462bc2ee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:13.121929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 640/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Sep 3 01:18:12 2022] PUT /identity/v3/projects/9f9529286db449018f980b53c7eb4e67/users/54b10e8ec2e244898f250cb987b6c7c8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:18:15.020277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cce8d-0826-47ee-9ccb-4b8e463e1200 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:15.020833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cce8d-0826-47ee-9ccb-4b8e463e1200 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:15.020833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cce8d-0826-47ee-9ccb-4b8e463e1200 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:15.022783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 634/1274] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:15.028301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:15.028801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:15.028905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:15.047514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:15.048705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:15.118221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 641/1275] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:15.390844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cce8d-0826-47ee-9ccb-4b8e463e1200 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:15.491391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cce8d-0826-47ee-9ccb-4b8e463e1200 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:15.508550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:15.524722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:15.525343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:15.525623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:15.525892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:15.535613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 03 01:18:15.540548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:15.542388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:15.543292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 635/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Sep 3 01:18:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:15.547886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:15.628318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:15.645782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d58fa-dc6d-4cbe-8192-2a16dd7af8c0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:15.680910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:15.681538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:15.681815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:15.682081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:15.682768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:15.683092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:15.684445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:15.692201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 642/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Sep 3 01:18:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:18:15.698201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:15.798792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad9a94-7480-4252-b9b2-1b7073cde646 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:15.832189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:15.832813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:15.833091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:15.833357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:15.843377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:15.844817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:15.859399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:15.860557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:15.861560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:15.861560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:15.861560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:15.861560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 636/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:15.867250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:15.964716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960b0c3-ef46-4e8f-90b5-79b3f77190a1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:15.999225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:15.999839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:16.000108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:16.000375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:16.001073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:16.001371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:16.002771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:16.016834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 643/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:16.023202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:16.054481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:16.124296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bde8f1-0a3e-4f8c-9b82-76e397432cce None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:16.158773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:16.159400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:16.159683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:16.159955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:16.167421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:16.168857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 03 01:18:16.180098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:16.181288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:16.182517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 637/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:16.187103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:16.191044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:16.287238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a54f7d-0df2-4b73-82ba-e8c785e68691 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:16.321453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:16.322149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:16.322527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:16.322801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:16.323550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:16.323848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:16.325287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:16.343004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 644/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Sep 3 01:18:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:18:16.348838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:16.439954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:16.449783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f845657-a602-4a7e-ac7f-2d9312b4f497 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:16.482633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:16.483245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:16.483524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:16.483791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:16.549990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:16.551422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:16.637587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:16.648120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 638/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Sep 3 01:18:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 03 01:18:16.654509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:16.714868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:16.758083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcc387-2f07-48ff-827f-7d7a3a2c5b58 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:16.791057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:16.791678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:16.791960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:16.792232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:16.802121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:16.804072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 03 01:18:16.819245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:16.820426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 03 01:18:16.821389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 03 01:18:16.821389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 03 01:18:16.821389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:16.821389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 645/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:16.826148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:16.922926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe835-faa7-4a8e-aa86-40899e8911a5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:16.957528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:16.958129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:16.958958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:16.959397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:16.960045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:16.960329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:16.961703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:16.972508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 639/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Sep 3 01:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:16.978886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:17.077743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad6061-597d-4d2b-adc8-eb7e068a4faf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:17.091256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:17.111098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:17.111718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:17.111997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:17.112268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:17.119335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:17.120773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 03 01:18:17.132114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:17.133226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 03 01:18:17.134419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 646/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Sep 3 01:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:17.139648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:17.189028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:17.236018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb11794-2d1c-408f-a048-78bb8cf518fc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:17.269461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:17.270067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:17.270362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:17.270645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:17.271779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:17.272087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:17.273524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87cc3c-30e5-4dfb-8202-a59551f4785b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:17.285828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 640/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Sep 3 01:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:17.292459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:17.392533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9aefcb-c0ae-4db1-8bb8-b27d403e48f8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:17.427047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:17.428589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:17.428875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:17.429150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:17.429880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] RBAC: Adding query filter params (role.id=00019edb8c5340468b5db8dc2c56fd73, scope.project.id=9f9529286db449018f980b53c7eb4e67, user.id=54b10e8ec2e244898f250cb987b6c7c8) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:17.430189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:17.431834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9493f06-84a9-4b22-b66e-c59e27dfa7da None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:17.437611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 647/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Sep 3 01:18:17 2022] GET /identity/v3/role_assignments?user.id=54b10e8ec2e244898f250cb987b6c7c8&scope.project.id=9f9529286db449018f980b53c7eb4e67&role.id=00019edb8c5340468b5db8dc2c56fd73 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:19.354802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10ae0d-ff3b-459c-a777-4f1115d8dba7 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:19.355335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10ae0d-ff3b-459c-a777-4f1115d8dba7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:19.355335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10ae0d-ff3b-459c-a777-4f1115d8dba7 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:19.356722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 641/1288] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:19.361752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:19.362129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:19.362310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:19.381121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:19.382311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:19.411377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:19.447960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 648/1289] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 03 01:18:19.726279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10ae0d-ff3b-459c-a777-4f1115d8dba7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:19.827092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10ae0d-ff3b-459c-a777-4f1115d8dba7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:19.838237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:19.860945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:19.861569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:19.861871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:19.862183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:19.862732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:19.864097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:19.873385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:19.874704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:19.875772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 642/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:18:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:19.880604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:19.979876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19819c2-3651-4045-8ac7-e7436b7186f0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:20.012008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:20.012598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:20.012875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:20.013141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:20.013760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:20.014055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:20.015467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb8f72-f11e-4988-b24b-92169d3f419d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:20.020137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 649/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Sep 3 01:18:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:20.026811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:20.126997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257fedf-0801-4369-9dc3-b7a07e687934 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:20.161259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:20.161963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:20.162246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:20.162553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:20.163180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:20.163485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:20.164867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bba0b-7790-4023-9b70-7ea8bab5601f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:20.169194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 643/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Sep 3 01:18:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:22.084395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a14b14-2af9-41b8-a75f-1aa37f642067 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:22.084959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a14b14-2af9-41b8-a75f-1aa37f642067 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:22.084959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a14b14-2af9-41b8-a75f-1aa37f642067 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:22.086287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 650/1293] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:22.091647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8dda-b8e9-44d0-9d43-ff9f75255b9e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:22.092076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8dda-b8e9-44d0-9d43-ff9f75255b9e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:22.092189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c8dda-b8e9-44d0-9d43-ff9f75255b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:22.111595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2d6c8dda-b8e9-44d0-9d43-ff9f75255b9e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:22.112778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2d6c8dda-b8e9-44d0-9d43-ff9f75255b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:22.141730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:22.179015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 644/1294] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:22.447342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a14b14-2af9-41b8-a75f-1aa37f642067 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:22.478569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:22.547948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a14b14-2af9-41b8-a75f-1aa37f642067 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:22.580858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9f299-c82b-4429-89e8-a5ce5d7c4e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:22.581449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9f299-c82b-4429-89e8-a5ce5d7c4e1d None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:22.581727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9f299-c82b-4429-89e8-a5ce5d7c4e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:22.581994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9f299-c82b-4429-89e8-a5ce5d7c4e1d None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:22.582693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9f299-c82b-4429-89e8-a5ce5d7c4e1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:22.584251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9f299-c82b-4429-89e8-a5ce5d7c4e1d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:22.658226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 651/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Sep 3 01:18:22 2022] POST /identity/v3/services => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:18:24.588724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9eb9-15f2-4a8d-85db-c1462166fe4f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:24.589285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9eb9-15f2-4a8d-85db-c1462166fe4f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:24.589285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9eb9-15f2-4a8d-85db-c1462166fe4f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:24.590657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 645/1296] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:24.595914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:24.596312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:24.596517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:24.615700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:24.616879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:24.688207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 652/1297] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:24.964188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec9eb9-15f2-4a8d-85db-c1462166fe4f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:25.064248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec9eb9-15f2-4a8d-85db-c1462166fe4f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:25.097537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:25.098162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:25.098473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:25.099349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:25.099930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:25.101313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:25.110413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:25.111691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:25.112848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 646/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:18:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:25.117694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:25.135540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:25.213441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6a4fc-9616-4e63-99e1-b6581f244c5e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:25.248186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:25.248797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:25.249082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:25.249357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:25.249986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:25.250290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:25.251714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:25.256875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 653/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Sep 3 01:18:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:25.262918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:25.295216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:25.363258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f912aa8-21e5-46e1-a317-0872027748bd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:25.397260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:25.397855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:25.398139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:25.398439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:25.399133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] RBAC: Adding query filter params (interface=public, service_id=f75cd87548204c66bb24f5fd7a3c2fe4, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:25.399442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:25.400816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092f56a-1fe7-4f0f-85ff-3313ba8c7145 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:25.405779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 647/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Sep 3 01:18:25 2022] GET /identity/v3/endpoints?service_id=f75cd87548204c66bb24f5fd7a3c2fe4&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:25.411543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:25.493095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:25.510795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27483a-3806-42bc-ad7a-3d80f06e1fa9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:25.545016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760751f-b469-4c67-9d75-cbb902763703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:25.545620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760751f-b469-4c67-9d75-cbb902763703 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:25.545926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760751f-b469-4c67-9d75-cbb902763703 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:25.546195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760751f-b469-4c67-9d75-cbb902763703 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:25.546919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760751f-b469-4c67-9d75-cbb902763703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:25.548436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760751f-b469-4c67-9d75-cbb902763703 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:25.555320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 654/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Sep 3 01:18:25 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:18:27.448927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda32d7-21a5-40c9-bd80-743634f5eb14 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:27.449457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda32d7-21a5-40c9-bd80-743634f5eb14 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:27.449457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda32d7-21a5-40c9-bd80-743634f5eb14 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:27.450885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 648/1302] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:18:27.456381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:27.456830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:27.456924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:27.476733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:18:27.477940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:18:27.552034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 655/1303] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:18:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 03 01:18:27.825651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda32d7-21a5-40c9-bd80-743634f5eb14 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:27.921261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:27.928560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda32d7-21a5-40c9-bd80-743634f5eb14 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:27.962179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:27.962826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:27.963109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:27.963380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:27.963957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:27.965390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 03 01:18:27.974693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 03 01:18:27.975879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Sep 03 01:18:27.977170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 649/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Sep 3 01:18:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 03 01:18:27.981773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:28.058649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:28.081050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b767dc-13a6-437f-b92f-874801c8cf35 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:28.115847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:28.116450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:28.116726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:28.116992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:28.117608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:18:28.117903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:28.119303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502e7e-cebe-4ab3-bde4-db63717c15fd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:28.124226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 656/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Sep 3 01:18:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:18:28.130398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:18:28.230156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e2c71-b308-48f3-a0b6-426e8c97a47f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:18:28.262206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59f596-27b3-471c-ae33-1d8e4ef83599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:18:28.262830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59f596-27b3-471c-ae33-1d8e4ef83599 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:18:28.263108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59f596-27b3-471c-ae33-1d8e4ef83599 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:18:28.263375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59f596-27b3-471c-ae33-1d8e4ef83599 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:18:28.264040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59f596-27b3-471c-ae33-1d8e4ef83599 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:18:28.265468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59f596-27b3-471c-ae33-1d8e4ef83599 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:18:28.343411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 650/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:18:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:18:38.132127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=30 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:18:38.338877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:22.711483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483e45-3003-427d-a154-af65a2b79fd1 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:22.712032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483e45-3003-427d-a154-af65a2b79fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:22.712032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a483e45-3003-427d-a154-af65a2b79fd1 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:22.713431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 657/1307] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:22.720284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b48857-22ce-4c70-9e6f-4c40ca0fddb1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:22.721288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b48857-22ce-4c70-9e6f-4c40ca0fddb1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:22.721535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b48857-22ce-4c70-9e6f-4c40ca0fddb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:22.846488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b7b48857-22ce-4c70-9e6f-4c40ca0fddb1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:22.846962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b7b48857-22ce-4c70-9e6f-4c40ca0fddb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:22.983106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 651/1308] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:23.282493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3802b36-2080-4f57-9755-5571e1d928aa None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:23.282702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3802b36-2080-4f57-9755-5571e1d928aa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:23.283545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3802b36-2080-4f57-9755-5571e1d928aa None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:23.284964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 658/1309] 10.222.0.13 () {54 vars in 965 bytes} [Sat Sep 3 01:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:23.290937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f593-5f9c-4690-b4fd-3b3747c53260 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:23.291294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f593-5f9c-4690-b4fd-3b3747c53260 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:23.291649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f593-5f9c-4690-b4fd-3b3747c53260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:23.371710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-dda8f593-5f9c-4690-b4fd-3b3747c53260 None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:23.373102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dda8f593-5f9c-4690-b4fd-3b3747c53260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:23.399210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:23.463009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 652/1310] 10.222.0.13 () {58 vars in 1062 bytes} [Sat Sep 3 01:24:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:23.473349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3802b36-2080-4f57-9755-5571e1d928aa None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:23.475226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f3802b36-2080-4f57-9755-5571e1d928aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:23.606350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3802b36-2080-4f57-9755-5571e1d928aa None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:23.687659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:23.688331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:23.688613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:23.688876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:23.691019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:23.747797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:23.804839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:23.806048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41e092-c164-49d2-a02e-49cfdad4e880 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:23.826731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 659/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:26.070443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0fa1-bccd-41c5-92f3-2b7a0cc35e49 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:26.070977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0fa1-bccd-41c5-92f3-2b7a0cc35e49 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:26.070977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986a0fa1-bccd-41c5-92f3-2b7a0cc35e49 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:26.072646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 653/1312] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:26.078322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8498b-14f8-4876-98ba-1d772f8480be None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:26.078880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8498b-14f8-4876-98ba-1d772f8480be None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:26.078974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8498b-14f8-4876-98ba-1d772f8480be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:26.172560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-58e8498b-14f8-4876-98ba-1d772f8480be None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:26.174493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-58e8498b-14f8-4876-98ba-1d772f8480be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:26.277840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 660/1313] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:26.563322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a0fa1-bccd-41c5-92f3-2b7a0cc35e49 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:26.565512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-986a0fa1-bccd-41c5-92f3-2b7a0cc35e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:26.668548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986a0fa1-bccd-41c5-92f3-2b7a0cc35e49 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:26.741602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:26.744218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:26.744839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:26.745119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:26.745389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:26.747114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:26.855351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:26.856793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ead835-3f79-490d-ae6b-9e6f63e7cbde service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:26.874954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 654/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:29.341351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b088113-b767-4c08-8509-046c2669bcac None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:29.341890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b088113-b767-4c08-8509-046c2669bcac None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:29.341890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b088113-b767-4c08-8509-046c2669bcac None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:29.343319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 661/1315] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:29.349007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04879f-4cb6-487d-a604-6b67bb41543c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:29.349387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04879f-4cb6-487d-a604-6b67bb41543c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:29.349565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04879f-4cb6-487d-a604-6b67bb41543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:29.414876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-de04879f-4cb6-487d-a604-6b67bb41543c None None] MFA Rules not processed for user `d0035d46b18a48c497b26eac6fb960ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:29.416085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-de04879f-4cb6-487d-a604-6b67bb41543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:29.466743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:29.512792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 655/1316] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:29.802634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b088113-b767-4c08-8509-046c2669bcac None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:29.804170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4b088113-b767-4c08-8509-046c2669bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:29.813918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:29.907117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b088113-b767-4c08-8509-046c2669bcac None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:29.985473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:29.986079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:29.986465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:29.986741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:29.988338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:30.051009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:30.097903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:30.099391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d075e-e293-4694-bf0f-11874aeb4e22 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:30.117105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 662/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:32.856653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0169d-3e0e-47b1-a0ea-34bfd7f5571d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:32.857159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0169d-3e0e-47b1-a0ea-34bfd7f5571d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:32.857254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0169d-3e0e-47b1-a0ea-34bfd7f5571d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:32.858572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 656/1318] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:32.865930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3527e8-a68a-4726-a77e-379c9f8209f3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:32.866383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3527e8-a68a-4726-a77e-379c9f8209f3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:32.866504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3527e8-a68a-4726-a77e-379c9f8209f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:32.933649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0f3527e8-a68a-4726-a77e-379c9f8209f3 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:32.934931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f3527e8-a68a-4726-a77e-379c9f8209f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:33.034702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 663/1319] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:33.336381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27f465-1c93-42f5-8006-ad495c9198a9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:33.336662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27f465-1c93-42f5-8006-ad495c9198a9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:33.336934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27f465-1c93-42f5-8006-ad495c9198a9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:33.338295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 657/1320] 10.222.0.13 () {54 vars in 965 bytes} [Sat Sep 3 01:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:33.344085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93789d-084a-4c29-ba4d-91d3fdd873d8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:33.344373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93789d-084a-4c29-ba4d-91d3fdd873d8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:33.344651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93789d-084a-4c29-ba4d-91d3fdd873d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:33.366515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:33.420427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0c93789d-084a-4c29-ba4d-91d3fdd873d8 None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:33.421646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0c93789d-084a-4c29-ba4d-91d3fdd873d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:33.494713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 664/1321] 10.222.0.13 () {58 vars in 1062 bytes} [Sat Sep 3 01:24:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:33.504123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27f465-1c93-42f5-8006-ad495c9198a9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:33.505683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9a27f465-1c93-42f5-8006-ad495c9198a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:33.609504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27f465-1c93-42f5-8006-ad495c9198a9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:33.683495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:33.684101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:33.684378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:33.684645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:33.686238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:33.693727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:33.800927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:33.802479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2957740-dd97-41fa-9df4-c6bcd20d6631 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:33.820543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 658/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:37.223185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61b5c-1ab9-4dfa-b83e-09e20a3402ff None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:37.223696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61b5c-1ab9-4dfa-b83e-09e20a3402ff None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:37.223696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61b5c-1ab9-4dfa-b83e-09e20a3402ff None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:37.225013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 665/1323] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:37.230261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de06d3b-b6f8-4054-806d-f210e7316660 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:37.230675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de06d3b-b6f8-4054-806d-f210e7316660 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:37.230955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de06d3b-b6f8-4054-806d-f210e7316660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:37.297057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8de06d3b-b6f8-4054-806d-f210e7316660 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:37.298880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8de06d3b-b6f8-4054-806d-f210e7316660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:37.357495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:37.397933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 659/1324] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:37.686238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61b5c-1ab9-4dfa-b83e-09e20a3402ff None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:37.687942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e0d61b5c-1ab9-4dfa-b83e-09e20a3402ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:37.811398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61b5c-1ab9-4dfa-b83e-09e20a3402ff None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:37.825085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:37.889771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:37.890422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:37.890707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:37.890975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:37.893195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:38.005996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:38.008050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90883029-d77c-4334-ba16-d3482ebe9e2f service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:38.030115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 666/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:40.073600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dc17e-eab7-4763-b0b2-3fac52168ef0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:40.074121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dc17e-eab7-4763-b0b2-3fac52168ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:40.074121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dc17e-eab7-4763-b0b2-3fac52168ef0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:40.075474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 660/1326] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:40.081419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e54f-11df-4873-885b-cbd9b192b164 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:40.081858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e54f-11df-4873-885b-cbd9b192b164 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:40.081982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884e54f-11df-4873-885b-cbd9b192b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:40.137877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:40.157959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a884e54f-11df-4873-885b-cbd9b192b164 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:40.159374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a884e54f-11df-4873-885b-cbd9b192b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:40.288198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 667/1327] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:24:40.644282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dc17e-eab7-4763-b0b2-3fac52168ef0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:40.645911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-661dc17e-eab7-4763-b0b2-3fac52168ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:40.750868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dc17e-eab7-4763-b0b2-3fac52168ef0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:40.838017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:40.838676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:40.838960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:40.839262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:40.840967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:40.862607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:40.958775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:40.960245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65cf0d-e67c-4341-8e62-74a7191c3b37 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:40.978736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 661/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:42.965614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8dcf7-f99b-4722-8c5f-c24c1248d23b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:42.966200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8dcf7-f99b-4722-8c5f-c24c1248d23b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:42.966200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a8dcf7-f99b-4722-8c5f-c24c1248d23b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:42.967568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 668/1329] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:42.973471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68add7-b635-48d6-995c-7fe8ef4623ee None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:42.973931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68add7-b635-48d6-995c-7fe8ef4623ee None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:42.974165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68add7-b635-48d6-995c-7fe8ef4623ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:43.042173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cb68add7-b635-48d6-995c-7fe8ef4623ee None None] MFA Rules not processed for user `d0035d46b18a48c497b26eac6fb960ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:43.043432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cb68add7-b635-48d6-995c-7fe8ef4623ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:43.115780 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:43.147640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 662/1330] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:43.432589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8dcf7-f99b-4722-8c5f-c24c1248d23b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:43.434165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-21a8dcf7-f99b-4722-8c5f-c24c1248d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:43.561755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a8dcf7-f99b-4722-8c5f-c24c1248d23b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:43.585535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:43.639732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:43.640340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:43.640619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:43.640886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:43.643418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:43.759198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:43.759557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d283e6-46a0-4d23-8606-bf89bb0779cc service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:43.778879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 669/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:46.169065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb56c65-a1a9-4f1d-a4bd-40e52b377a99 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:46.169586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb56c65-a1a9-4f1d-a4bd-40e52b377a99 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:46.169586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb56c65-a1a9-4f1d-a4bd-40e52b377a99 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:46.170945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 663/1332] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:46.176767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ce85d-2964-49b8-be80-73618d208e2c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:46.177188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ce85d-2964-49b8-be80-73618d208e2c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:46.177332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ce85d-2964-49b8-be80-73618d208e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:46.240190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:46.242903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-428ce85d-2964-49b8-be80-73618d208e2c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:46.244116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-428ce85d-2964-49b8-be80-73618d208e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:46.341727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 670/1333] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:46.625696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb56c65-a1a9-4f1d-a4bd-40e52b377a99 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:46.627803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4fb56c65-a1a9-4f1d-a4bd-40e52b377a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:46.730352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb56c65-a1a9-4f1d-a4bd-40e52b377a99 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:46.802306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:46.803203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:46.803203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:46.803468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:46.805057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:46.831769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:46.913535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:46.915014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ac2b-8b99-4c77-8d26-a218684d8cfe service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:46.933550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 664/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:48.915528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7144-0fa7-460b-a0c8-f14407b98321 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:48.916071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7144-0fa7-460b-a0c8-f14407b98321 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:48.916071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7144-0fa7-460b-a0c8-f14407b98321 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:48.917465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 671/1335] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:48.923420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba60d-4627-40e2-8bb2-3481f3b89a3e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:48.928890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba60d-4627-40e2-8bb2-3481f3b89a3e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:48.928890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba60d-4627-40e2-8bb2-3481f3b89a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:48.990566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0c0ba60d-4627-40e2-8bb2-3481f3b89a3e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:48.991774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0c0ba60d-4627-40e2-8bb2-3481f3b89a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:49.066479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:49.089186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 665/1336] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:49.377951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f7144-0fa7-460b-a0c8-f14407b98321 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:49.379534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-927f7144-0fa7-460b-a0c8-f14407b98321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:49.484248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f7144-0fa7-460b-a0c8-f14407b98321 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:49.512516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:49.558569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:49.559203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:49.559464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:49.559731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:49.561332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:49.671386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:49.673391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dff606-bf75-4c1d-98e5-01bb4f3a7c3e service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:49.692445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 672/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:52.212881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d28b-b3d6-4e12-8fe6-19a0cd9504c8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:52.213387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d28b-b3d6-4e12-8fe6-19a0cd9504c8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:52.213387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d28b-b3d6-4e12-8fe6-19a0cd9504c8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:52.214745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 666/1338] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:52.219963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7976a0-c572-4123-bc24-bb60d064a87b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:52.220388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7976a0-c572-4123-bc24-bb60d064a87b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:52.220499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7976a0-c572-4123-bc24-bb60d064a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:52.285490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8e7976a0-c572-4123-bc24-bb60d064a87b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:52.286742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8e7976a0-c572-4123-bc24-bb60d064a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:52.294886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:52.384520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 673/1339] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:52.666547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d28b-b3d6-4e12-8fe6-19a0cd9504c8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:52.668090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b480d28b-b3d6-4e12-8fe6-19a0cd9504c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:52.771056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d28b-b3d6-4e12-8fe6-19a0cd9504c8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:52.844141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:52.844987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:52.845275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:52.845545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:52.847162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:52.883362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:52.958062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:52.959659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd9b45-24e8-4cde-8b77-afd78b65f23c service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:52.979867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 667/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:24:57.158037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76258b-1a3d-478b-8151-ad65a754c6d2 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:57.158643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76258b-1a3d-478b-8151-ad65a754c6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:57.158747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76258b-1a3d-478b-8151-ad65a754c6d2 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:57.160082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 674/1341] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:24:57.165376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab5a76-5e79-4879-a408-27cff1ba80fb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:57.165798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab5a76-5e79-4879-a408-27cff1ba80fb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:57.165947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab5a76-5e79-4879-a408-27cff1ba80fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:57.232309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-caab5a76-5e79-4879-a408-27cff1ba80fb None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:24:57.233527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-caab5a76-5e79-4879-a408-27cff1ba80fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:57.324020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:57.335512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 668/1342] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:24:57.620857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76258b-1a3d-478b-8151-ad65a754c6d2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:24:57.622439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7a76258b-1a3d-478b-8151-ad65a754c6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:57.727138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76258b-1a3d-478b-8151-ad65a754c6d2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:24:57.765609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:24:57.805504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:24:57.806122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:24:57.806433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:24:57.806708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:24:57.808363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:24:57.924362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:24:57.925738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a5099-daca-4611-9348-0c7dcfada550 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:24:57.946610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 675/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:00.896711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a3ea-2b2d-432a-8752-264a56619d9f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:00.897213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a3ea-2b2d-432a-8752-264a56619d9f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:00.897213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943a3ea-2b2d-432a-8752-264a56619d9f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:00.898595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 669/1344] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:00.904068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d05f6-1e06-4b34-ae5b-e20e7e0fb8f6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:00.904501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d05f6-1e06-4b34-ae5b-e20e7e0fb8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:00.904738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d05f6-1e06-4b34-ae5b-e20e7e0fb8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:00.973516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-158d05f6-1e06-4b34-ae5b-e20e7e0fb8f6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:00.975180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-158d05f6-1e06-4b34-ae5b-e20e7e0fb8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:00.996641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:01.081046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 676/1345] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:01.374316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a3ea-2b2d-432a-8752-264a56619d9f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:01.376413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a943a3ea-2b2d-432a-8752-264a56619d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:01.490428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943a3ea-2b2d-432a-8752-264a56619d9f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:01.566788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:01.567396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:01.567674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:01.567945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:01.569605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:01.614919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:01.686149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:01.687540 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d4d58-b8c1-4c51-98d7-88bbf18387f0 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:01.706031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 670/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:06.063511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981c2af-30db-4386-b1b9-b13390b066ec None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:06.064181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981c2af-30db-4386-b1b9-b13390b066ec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:06.064181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981c2af-30db-4386-b1b9-b13390b066ec None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:06.065793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 677/1347] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:06.070945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822662-462f-4041-b2a7-c893023b6ada None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:06.071376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822662-462f-4041-b2a7-c893023b6ada None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:06.071501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822662-462f-4041-b2a7-c893023b6ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:06.143980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-00822662-462f-4041-b2a7-c893023b6ada None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:06.145214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-00822662-462f-4041-b2a7-c893023b6ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:06.245877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:06.248357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 671/1348] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:06.543765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981c2af-30db-4386-b1b9-b13390b066ec None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:06.546394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3981c2af-30db-4386-b1b9-b13390b066ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:06.665540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981c2af-30db-4386-b1b9-b13390b066ec None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:06.714769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:06.740573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:06.741425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:06.741537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:06.741705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:06.743354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:06.856651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:06.858019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c08e3-bbd4-4c7e-9dcf-3e92cdf52b5b service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:06.876705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 678/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:11.850298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cd98e-90c4-4262-a22f-c3605a633674 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:11.850858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cd98e-90c4-4262-a22f-c3605a633674 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:11.851147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cd98e-90c4-4262-a22f-c3605a633674 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:11.852860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 672/1350] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:11.857699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bfcbe-d358-4dfd-b346-5559f079277d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:11.858127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bfcbe-d358-4dfd-b346-5559f079277d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:11.858303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bfcbe-d358-4dfd-b346-5559f079277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:11.961197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-861bfcbe-d358-4dfd-b346-5559f079277d None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:11.961197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-861bfcbe-d358-4dfd-b346-5559f079277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:12.004084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:12.150088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 679/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:25:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:16.514863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869d35-523f-4626-a251-7c6dae717646 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:16.516442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869d35-523f-4626-a251-7c6dae717646 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:16.516825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0869d35-523f-4626-a251-7c6dae717646 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:16.518300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 673/1352] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:16.523738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd74e7-cbea-4dc3-a3cd-aa8a9980585a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:16.524184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd74e7-cbea-4dc3-a3cd-aa8a9980585a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:16.524404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd74e7-cbea-4dc3-a3cd-aa8a9980585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:16.604938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8dbd74e7-cbea-4dc3-a3cd-aa8a9980585a None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:16.606139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8dbd74e7-cbea-4dc3-a3cd-aa8a9980585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:16.683328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:16.722481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 680/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:25:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:18.083074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeabe4-b64b-4ef2-8669-ccfb89af0c4d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:18.083627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeabe4-b64b-4ef2-8669-ccfb89af0c4d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:18.083627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeabe4-b64b-4ef2-8669-ccfb89af0c4d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:18.084879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 674/1354] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:18.089880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fde80-4d30-4d9e-9742-0c8414b02be2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:18.090330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fde80-4d30-4d9e-9742-0c8414b02be2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:18.090535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fde80-4d30-4d9e-9742-0c8414b02be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:18.167905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c13fde80-4d30-4d9e-9742-0c8414b02be2 None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:18.169123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c13fde80-4d30-4d9e-9742-0c8414b02be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:18.280909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 681/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:25:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:21.606232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc6f02-f478-431d-a95a-1e95ee69e6b8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:21.606797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc6f02-f478-431d-a95a-1e95ee69e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:21.606797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc6f02-f478-431d-a95a-1e95ee69e6b8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:21.608076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 675/1356] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:21.614165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718086-787f-45b1-8d8b-4b04670fc062 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:21.614582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718086-787f-45b1-8d8b-4b04670fc062 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:21.614966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8718086-787f-45b1-8d8b-4b04670fc062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:21.666203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:21.703633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a8718086-787f-45b1-8d8b-4b04670fc062 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:21.703633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a8718086-787f-45b1-8d8b-4b04670fc062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:21.842385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 682/1357] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:22.162835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e7394-c9dc-4046-8832-c9499deed310 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:22.163508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e7394-c9dc-4046-8832-c9499deed310 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:22.164100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e7394-c9dc-4046-8832-c9499deed310 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:22.166532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 676/1358] 10.222.0.13 () {54 vars in 962 bytes} [Sat Sep 3 01:25:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:22.173313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de3cf8-f378-400c-b894-0cc266bd085d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:22.173609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de3cf8-f378-400c-b894-0cc266bd085d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:22.173890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de3cf8-f378-400c-b894-0cc266bd085d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:22.271985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d3de3cf8-f378-400c-b894-0cc266bd085d None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:22.273113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d3de3cf8-f378-400c-b894-0cc266bd085d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:22.326818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:22.396354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 683/1359] 10.222.0.13 () {58 vars in 1059 bytes} [Sat Sep 3 01:25:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:22.404478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e7394-c9dc-4046-8832-c9499deed310 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:22.406669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a94e7394-c9dc-4046-8832-c9499deed310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:22.564740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e7394-c9dc-4046-8832-c9499deed310 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:22.642411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:22.684123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:22.684744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:22.685025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:22.685301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:22.687588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:22.780203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:22.781963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59b4e-e7ee-4dbe-9c76-48314b1783ab service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:22.805025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 677/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:24.960139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa0f3c-1d97-4aea-ae4f-17611089e135 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:24.960679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa0f3c-1d97-4aea-ae4f-17611089e135 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:24.960679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa0f3c-1d97-4aea-ae4f-17611089e135 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:24.962244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 684/1361] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:24.967771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa16efa-bdb1-4097-944a-72a59e8a4869 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:24.970067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa16efa-bdb1-4097-944a-72a59e8a4869 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:24.970067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa16efa-bdb1-4097-944a-72a59e8a4869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:25.039361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-caa16efa-bdb1-4097-944a-72a59e8a4869 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:25.040607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-caa16efa-bdb1-4097-944a-72a59e8a4869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:25.080714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:25.141794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 678/1362] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:25.435657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa0f3c-1d97-4aea-ae4f-17611089e135 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:25.437825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-19aa0f3c-1d97-4aea-ae4f-17611089e135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:25.599497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa0f3c-1d97-4aea-ae4f-17611089e135 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:25.608780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:25.722888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:25.723510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:25.723791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:25.724061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:25.725697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:25.840981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:25.842646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc794e5-ee24-40f0-9de4-3975615872ba service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:25.867207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 685/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:28.307564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df802118-3ab5-491d-9d3c-c6cb70130ad0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:28.308168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df802118-3ab5-491d-9d3c-c6cb70130ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:28.308168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df802118-3ab5-491d-9d3c-c6cb70130ad0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:28.309468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 679/1364] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:28.314953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610246cf-cfa8-4af8-8f38-8b1c39d18528 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:28.315382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610246cf-cfa8-4af8-8f38-8b1c39d18528 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:28.315581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610246cf-cfa8-4af8-8f38-8b1c39d18528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:28.332141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:28.385634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-610246cf-cfa8-4af8-8f38-8b1c39d18528 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:28.386941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-610246cf-cfa8-4af8-8f38-8b1c39d18528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:28.496018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 686/1365] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:28.789348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63e0f5-b5c1-4cfd-83d7-ee9873ec7454 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:28.789635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63e0f5-b5c1-4cfd-83d7-ee9873ec7454 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:28.790113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63e0f5-b5c1-4cfd-83d7-ee9873ec7454 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:28.791658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 680/1366] 10.222.0.13 () {54 vars in 962 bytes} [Sat Sep 3 01:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:28.800052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885711a-2773-4baf-a9dd-81e1e74bb179 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:28.800052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885711a-2773-4baf-a9dd-81e1e74bb179 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:28.800052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885711a-2773-4baf-a9dd-81e1e74bb179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:28.883282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2885711a-2773-4baf-a9dd-81e1e74bb179 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:28.884834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2885711a-2773-4baf-a9dd-81e1e74bb179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:28.926875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:29.017513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 687/1367] 10.222.0.13 () {58 vars in 1059 bytes} [Sat Sep 3 01:25:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:29.026147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63e0f5-b5c1-4cfd-83d7-ee9873ec7454 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:29.027995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bd63e0f5-b5c1-4cfd-83d7-ee9873ec7454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:29.241235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63e0f5-b5c1-4cfd-83d7-ee9873ec7454 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:29.269129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:29.391326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e615be-0447-405e-af31-7e182e271005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:29.392055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e615be-0447-405e-af31-7e182e271005 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:29.392427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e615be-0447-405e-af31-7e182e271005 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:29.392792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e615be-0447-405e-af31-7e182e271005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:29.394799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-27e615be-0447-405e-af31-7e182e271005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:29.506375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e615be-0447-405e-af31-7e182e271005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:29.506375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e615be-0447-405e-af31-7e182e271005 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:29.525233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 681/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:29.902487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c38ff-abbe-4379-9cf7-b2242c524fd8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:29.903023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c38ff-abbe-4379-9cf7-b2242c524fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:29.903023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c38ff-abbe-4379-9cf7-b2242c524fd8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:29.904576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 688/1369] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:29.910369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e179ebc-19aa-41a5-80fe-113b55b44a1b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:29.910687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e179ebc-19aa-41a5-80fe-113b55b44a1b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:29.910956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e179ebc-19aa-41a5-80fe-113b55b44a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:29.946892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:29.984620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5e179ebc-19aa-41a5-80fe-113b55b44a1b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:29.985833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5e179ebc-19aa-41a5-80fe-113b55b44a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:30.092810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 682/1370] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:30.677314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933948-cbaa-45ad-bc54-75a9794b3c56 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:30.677314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933948-cbaa-45ad-bc54-75a9794b3c56 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:30.677314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10933948-cbaa-45ad-bc54-75a9794b3c56 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:30.678445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 689/1371] 10.222.0.13 () {54 vars in 964 bytes} [Sat Sep 3 01:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:30.683672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2754b-4234-4ef2-bc6e-98df6a913c44 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:30.684182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2754b-4234-4ef2-bc6e-98df6a913c44 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:30.684182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2754b-4234-4ef2-bc6e-98df6a913c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:30.765521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6eb2754b-4234-4ef2-bc6e-98df6a913c44 None None] MFA Rules not processed for user `edb970ecb499474f87e02b9ee49c8c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:30.766792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6eb2754b-4234-4ef2-bc6e-98df6a913c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:30.820831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:30.854210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 683/1372] 10.222.0.13 () {58 vars in 1061 bytes} [Sat Sep 3 01:25:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:30.864138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933948-cbaa-45ad-bc54-75a9794b3c56 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:30.865884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-10933948-cbaa-45ad-bc54-75a9794b3c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:30.983073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10933948-cbaa-45ad-bc54-75a9794b3c56 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:31.027506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:31.067368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:31.068021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:31.068310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:31.068584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:31.070213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:31.196239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:31.197622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6b191-fefd-4577-8e37-27c371bfd77c service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:31.218836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 690/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:31.718718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95d1fe-6557-4d9c-811c-ae56f2b9e176 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:31.719299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95d1fe-6557-4d9c-811c-ae56f2b9e176 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:31.719299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95d1fe-6557-4d9c-811c-ae56f2b9e176 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:31.720605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 684/1374] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:31.725915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e7d5-f88f-4d16-8a9d-3e7b165a8710 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:31.726204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e7d5-f88f-4d16-8a9d-3e7b165a8710 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:31.726510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8e7d5-f88f-4d16-8a9d-3e7b165a8710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:31.792179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fde8e7d5-f88f-4d16-8a9d-3e7b165a8710 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:31.793384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fde8e7d5-f88f-4d16-8a9d-3e7b165a8710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:31.816104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:31.893324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 691/1375] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:32.201085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95d1fe-6557-4d9c-811c-ae56f2b9e176 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:32.202749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ee95d1fe-6557-4d9c-811c-ae56f2b9e176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:32.360159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95d1fe-6557-4d9c-811c-ae56f2b9e176 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:32.411571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:32.473925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:32.474565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:32.474848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:32.475120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:32.476721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:32.566235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:32.567897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72667444-dbd8-4d54-9186-5ec0ade5856a service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:32.586102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 685/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:34.695142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb9452-e68f-4194-b7f5-7116a91f8fb0 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:34.695582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb9452-e68f-4194-b7f5-7116a91f8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:34.695695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb9452-e68f-4194-b7f5-7116a91f8fb0 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:34.697048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 692/1377] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:34.702511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85eaab-32be-4a92-93e1-3f45cf7e0f90 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:34.702998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85eaab-32be-4a92-93e1-3f45cf7e0f90 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:34.703346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85eaab-32be-4a92-93e1-3f45cf7e0f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:34.778432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1d85eaab-32be-4a92-93e1-3f45cf7e0f90 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:34.779640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1d85eaab-32be-4a92-93e1-3f45cf7e0f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:34.787815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:34.903061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 686/1378] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:35.215286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb9452-e68f-4194-b7f5-7116a91f8fb0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:35.216815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2afb9452-e68f-4194-b7f5-7116a91f8fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:35.374395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb9452-e68f-4194-b7f5-7116a91f8fb0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:35.378465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:35.497171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:35.497719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:35.497996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:35.498265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:35.500641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:35.596612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:35.598246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7395571-57c2-497f-9268-8412dfb634c7 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:35.617522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 693/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:37.667954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ef9cd-1c70-4b0b-a5f9-3cc24dc64f95 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:37.668501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ef9cd-1c70-4b0b-a5f9-3cc24dc64f95 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:37.668501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ef9cd-1c70-4b0b-a5f9-3cc24dc64f95 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:37.670004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 687/1380] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:25:37.675856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e524-8a49-4093-8bc5-15f89d57a132 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:37.676310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e524-8a49-4093-8bc5-15f89d57a132 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:37.676390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885e524-8a49-4093-8bc5-15f89d57a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:37.688453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:37.745831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4885e524-8a49-4093-8bc5-15f89d57a132 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:37.747106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4885e524-8a49-4093-8bc5-15f89d57a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:37.873992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 694/1381] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:38.179576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ef9cd-1c70-4b0b-a5f9-3cc24dc64f95 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:38.184132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6a5ef9cd-1c70-4b0b-a5f9-3cc24dc64f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:38.329675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:38.339718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ef9cd-1c70-4b0b-a5f9-3cc24dc64f95 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:38.460057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:38.460668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:38.460950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:38.461223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:38.462994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:38.562020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:38.563945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4bc44-09a9-4cf0-a586-ee67eb57791e service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:38.575360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:38.586640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 688/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:41.129027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23dbe-16c5-4feb-97a8-b9bdd61cc8b3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:41.129537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23dbe-16c5-4feb-97a8-b9bdd61cc8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:41.129537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23dbe-16c5-4feb-97a8-b9bdd61cc8b3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:41.131138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 695/1383] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:41.136856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9017-d621-4706-897b-33dfbfbf3e56 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:41.137693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9017-d621-4706-897b-33dfbfbf3e56 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:41.138027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a9017-d621-4706-897b-33dfbfbf3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:41.216571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9e1a9017-d621-4706-897b-33dfbfbf3e56 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:41.218181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9e1a9017-d621-4706-897b-33dfbfbf3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:41.325127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 689/1384] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:41.698185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23dbe-16c5-4feb-97a8-b9bdd61cc8b3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:41.699755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dbf23dbe-16c5-4feb-97a8-b9bdd61cc8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:41.792270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:41.859410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23dbe-16c5-4feb-97a8-b9bdd61cc8b3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:41.993057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:41.994015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:41.994368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:41.994734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:41.996755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:42.055581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:42.099879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:42.102106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a325f-fae9-4204-8c5d-cce07663ec1b service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:42.132206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 696/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:44.081228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481031f8-9ae3-4c7a-b924-468b8af7e63f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:44.081820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481031f8-9ae3-4c7a-b924-468b8af7e63f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:44.081820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481031f8-9ae3-4c7a-b924-468b8af7e63f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:44.083161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 690/1386] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:25:44.088901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d687-f41e-469a-b0fd-fe36824fd172 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:44.089324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d687-f41e-469a-b0fd-fe36824fd172 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:44.089595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d687-f41e-469a-b0fd-fe36824fd172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:44.169408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a242d687-f41e-469a-b0fd-fe36824fd172 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:44.170902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a242d687-f41e-469a-b0fd-fe36824fd172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:44.279255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 697/1387] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:44.725217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481031f8-9ae3-4c7a-b924-468b8af7e63f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:44.726806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-481031f8-9ae3-4c7a-b924-468b8af7e63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:44.808874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:44.882544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481031f8-9ae3-4c7a-b924-468b8af7e63f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:44.997517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:44.998125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:44.998430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:45.000085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:45.001672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:45.041327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:45.095022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:45.096649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685d262-7d37-4f40-8366-8ee09e87136b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:45.114601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 691/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:46.616086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b606bc-0cda-46a4-ba88-dd75499a5fee None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:46.616660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b606bc-0cda-46a4-ba88-dd75499a5fee None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:46.616660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b606bc-0cda-46a4-ba88-dd75499a5fee None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:46.617974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 698/1389] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:46.623142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a966795-7e87-4341-9cdf-bc6c350e9325 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:46.624066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a966795-7e87-4341-9cdf-bc6c350e9325 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:46.624313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a966795-7e87-4341-9cdf-bc6c350e9325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:46.694487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5a966795-7e87-4341-9cdf-bc6c350e9325 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:46.696193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5a966795-7e87-4341-9cdf-bc6c350e9325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:46.814375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 692/1390] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:47.117721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dca5c2-170e-4bf8-8f66-34e64caf7f36 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:47.118012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dca5c2-170e-4bf8-8f66-34e64caf7f36 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:47.118291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dca5c2-170e-4bf8-8f66-34e64caf7f36 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:47.119670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 699/1391] 10.222.0.13 () {54 vars in 961 bytes} [Sat Sep 3 01:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:47.125500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:47.125781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:47.126052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:47.142189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:47.172609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f125af-18ba-40bf-bbac-92b17ee45643 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:47.172901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f125af-18ba-40bf-bbac-92b17ee45643 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:47.173189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f125af-18ba-40bf-bbac-92b17ee45643 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:47.174609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 700/1392] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:25:47.179675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:47.179967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:47.180247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:47.204259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] MFA Rules not processed for user `cbc9f7e49e5946f48df125d4efd80dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:47.205473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:47.215354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:47.248013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:47.249831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:47.285152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 693/1393] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Sep 3 01:25:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:47.292777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:47.300484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07002247-40b0-4e95-89e8-afecf0e5da54 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:47.351462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 701/1394] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:47.376457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:47.377070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:47.377352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:47.377660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:47.379284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:47.420327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:47.511793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:47.513234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a73d9-749a-4817-acc6-28381a108f11 service glance] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:47.533679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 694/1395] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:25:47.689069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:47.690285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:47.809967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:47.818584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca47cfc-b4b7-453b-a716-1ed711de8b56 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:47.938172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:47.938736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:47.939017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:47.939287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:47.940959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:48.038109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:48.039809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d858b-4a2a-46b4-85ef-1dd1bca50ec4 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:48.056656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:48.059660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 702/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:25:48.418615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfa9fe-0eb6-4ade-9082-f82c8a9f165e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:48.419093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfa9fe-0eb6-4ade-9082-f82c8a9f165e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:48.419287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdfa9fe-0eb6-4ade-9082-f82c8a9f165e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:48.488906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ecdfa9fe-0eb6-4ade-9082-f82c8a9f165e None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:48.490124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ecdfa9fe-0eb6-4ade-9082-f82c8a9f165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:48.573522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:48.580172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 695/1397] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:25:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:48.626256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10fffa-7b7a-4846-b4d0-570c3ed7f41c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:48.626571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10fffa-7b7a-4846-b4d0-570c3ed7f41c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:48.626864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10fffa-7b7a-4846-b4d0-570c3ed7f41c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:48.628229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 703/1398] 10.222.0.13 () {54 vars in 943 bytes} [Sat Sep 3 01:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:48.634987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c83b4-961b-4883-a4a8-975f5d7746ab None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:48.635272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c83b4-961b-4883-a4a8-975f5d7746ab None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:48.635543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c83b4-961b-4883-a4a8-975f5d7746ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:48.715008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-595c83b4-961b-4883-a4a8-975f5d7746ab None None] MFA Rules not processed for user `0711b98a2c214b7cb807f2d211b628c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:48.716522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-595c83b4-961b-4883-a4a8-975f5d7746ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:48.832476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 696/1399] 10.222.0.13 () {58 vars in 1040 bytes} [Sat Sep 3 01:25:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:48.841843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10fffa-7b7a-4846-b4d0-570c3ed7f41c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:48.843977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7b10fffa-7b7a-4846-b4d0-570c3ed7f41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:48.984045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10fffa-7b7a-4846-b4d0-570c3ed7f41c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:49.071718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:49.091339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:49.091924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:49.092449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:49.092771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:49.094484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:49.181281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:49.182909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d01cd-5f13-4760-a1a6-4e1df4fd625f service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:49.191473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 704/1400] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:25:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:25:50.139684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06b9f6-0605-4489-bd1d-2ca6a0400d20 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:50.140309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06b9f6-0605-4489-bd1d-2ca6a0400d20 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:50.140309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06b9f6-0605-4489-bd1d-2ca6a0400d20 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:50.141942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 697/1401] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:50.147298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c05e-b24a-4936-9b37-4789de61345d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:50.147720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c05e-b24a-4936-9b37-4789de61345d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:50.147826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c05e-b24a-4936-9b37-4789de61345d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:50.230271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a900c05e-b24a-4936-9b37-4789de61345d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:50.231720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a900c05e-b24a-4936-9b37-4789de61345d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:50.307774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:50.339543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 705/1402] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:25:50.639644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06b9f6-0605-4489-bd1d-2ca6a0400d20 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:50.641199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7e06b9f6-0605-4489-bd1d-2ca6a0400d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:50.707284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:50.809535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06b9f6-0605-4489-bd1d-2ca6a0400d20 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:50.939714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:50.940368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:50.940658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:50.940928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:50.942608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:50.960033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:51.045972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:51.047981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552cb06-f0cd-492c-be97-2f17e9198c0c service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:51.068871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 698/1403] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:53.157328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb7c8-109a-4df4-8f2a-8ebe88d3acec None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:53.157879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb7c8-109a-4df4-8f2a-8ebe88d3acec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:53.157879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb7c8-109a-4df4-8f2a-8ebe88d3acec None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:53.159315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 706/1404] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:53.164820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51047f-0377-4148-bafd-fd50ee5786ad None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:53.165320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51047f-0377-4148-bafd-fd50ee5786ad None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:53.165320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51047f-0377-4148-bafd-fd50ee5786ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:53.234147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8e51047f-0377-4148-bafd-fd50ee5786ad None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:53.235370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8e51047f-0377-4148-bafd-fd50ee5786ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:53.338122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:53.347776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 699/1405] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:53.644014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fb7c8-109a-4df4-8f2a-8ebe88d3acec None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:53.645635 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-351fb7c8-109a-4df4-8f2a-8ebe88d3acec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:53.808160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fb7c8-109a-4df4-8f2a-8ebe88d3acec None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:53.875128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:53.941583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:53.942488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:53.942931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:53.943262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:53.945778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:54.052490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:54.054120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343f3a01-89bf-406d-9782-95765b4a0caa service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:54.075152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 707/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:25:56.203501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4348-7908-4431-a90c-4a0ee2b07324 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:56.203501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4348-7908-4431-a90c-4a0ee2b07324 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:56.205877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4348-7908-4431-a90c-4a0ee2b07324 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:56.205877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 700/1407] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:25:56.211558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a8f9d-0401-4125-a541-e5ca40aee74b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:56.211997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a8f9d-0401-4125-a541-e5ca40aee74b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:56.212095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a8f9d-0401-4125-a541-e5ca40aee74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:56.298021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-658a8f9d-0401-4125-a541-e5ca40aee74b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:25:56.299339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-658a8f9d-0401-4125-a541-e5ca40aee74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:56.316913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:56.436685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 708/1408] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:25:56.725581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4348-7908-4431-a90c-4a0ee2b07324 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:25:56.727253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c59e4348-7908-4431-a90c-4a0ee2b07324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:56.894669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4348-7908-4431-a90c-4a0ee2b07324 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:25:56.973771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:25:57.025942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:25:57.026858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:25:57.027140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:25:57.027410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:25:57.029067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:25:57.130165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:25:57.131817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2fb7f-e5cb-49d1-a858-d6973eb7864e service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:25:57.153069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 701/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:06.490731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=37 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:07.150434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=65 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:12.749492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dedd2-f9fc-4706-9cef-d484e9c7a2a5 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:12.750054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dedd2-f9fc-4706-9cef-d484e9c7a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:12.750054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1dedd2-f9fc-4706-9cef-d484e9c7a2a5 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:12.751469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 709/1410] 10.222.0.13 () {54 vars in 918 bytes} [Sat Sep 3 01:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:12.756618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a84209-b172-4744-a084-96ffaaca2a9b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:12.757050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a84209-b172-4744-a084-96ffaaca2a9b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:12.757171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a84209-b172-4744-a084-96ffaaca2a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:12.834140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-20a84209-b172-4744-a084-96ffaaca2a9b None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:12.835430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-20a84209-b172-4744-a084-96ffaaca2a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:12.955102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 702/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Sat Sep 3 01:26:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:26:12.969865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de0ccf-46c5-404c-b93d-578181f8e49c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:12.970154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de0ccf-46c5-404c-b93d-578181f8e49c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:12.970465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de0ccf-46c5-404c-b93d-578181f8e49c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:12.971839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 710/1412] 10.222.0.13 () {54 vars in 918 bytes} [Sat Sep 3 01:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:12.976370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ef38-232c-4074-b000-ff97dc89256c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:12.976664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ef38-232c-4074-b000-ff97dc89256c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:12.976963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ef38-232c-4074-b000-ff97dc89256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:13.050849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:13.053762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b949ef38-232c-4074-b000-ff97dc89256c None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:13.055019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b949ef38-232c-4074-b000-ff97dc89256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:13.176302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 703/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Sat Sep 3 01:26:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:15.615131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e8f7-2952-4d32-ae41-cebabf7f474d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:15.615642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e8f7-2952-4d32-ae41-cebabf7f474d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:15.615642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e8f7-2952-4d32-ae41-cebabf7f474d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:15.616927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:15.622533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5458b86-3133-465d-8ba8-9a12ed02ba6c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:15.623000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5458b86-3133-465d-8ba8-9a12ed02ba6c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:15.623183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5458b86-3133-465d-8ba8-9a12ed02ba6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:15.689287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b5458b86-3133-465d-8ba8-9a12ed02ba6c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:15.690544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b5458b86-3133-465d-8ba8-9a12ed02ba6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:15.744121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:15.790437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:16.082953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e8f7-2952-4d32-ae41-cebabf7f474d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:16.084502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9289e8f7-2952-4d32-ae41-cebabf7f474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:16.228559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e8f7-2952-4d32-ae41-cebabf7f474d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:16.299856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:16.339770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:16.340309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:16.340592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:16.340862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:16.343091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:16.432006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:16.433621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550f621-186f-4180-80dc-21a836240ddb service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:16.452396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 712/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:17.352790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb17cb5-7fc9-4a5a-b9f5-a12c0d229bda None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:17.353443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb17cb5-7fc9-4a5a-b9f5-a12c0d229bda None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:17.353443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb17cb5-7fc9-4a5a-b9f5-a12c0d229bda None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:17.354748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:17.360137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896ee50-71b3-4e4a-806a-4aa34f3b45bf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:17.360582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896ee50-71b3-4e4a-806a-4aa34f3b45bf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:17.360702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896ee50-71b3-4e4a-806a-4aa34f3b45bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:17.437576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1896ee50-71b3-4e4a-806a-4aa34f3b45bf None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:17.438811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1896ee50-71b3-4e4a-806a-4aa34f3b45bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:17.465167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:17.533548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:17.682736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb17cb5-7fc9-4a5a-b9f5-a12c0d229bda None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:17.684190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9eb17cb5-7fc9-4a5a-b9f5-a12c0d229bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:17.830778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb17cb5-7fc9-4a5a-b9f5-a12c0d229bda None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:17.871392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:17.945831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:17.946376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:17.946659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:17.946927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:17.948557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:18.038897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:18.040493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8325897-9011-4a49-ad22-c1e454da2b5b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:18.058125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 706/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:18.939866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dbd87-8798-46eb-acf7-f35edf219a66 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:18.940395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dbd87-8798-46eb-acf7-f35edf219a66 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:18.940395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dbd87-8798-46eb-acf7-f35edf219a66 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:18.941743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:18.946663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e47606-2829-4a1f-9313-5951bdc3a89f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:18.947111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e47606-2829-4a1f-9313-5951bdc3a89f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:18.947233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e47606-2829-4a1f-9313-5951bdc3a89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:19.000848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:19.023343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-49e47606-2829-4a1f-9313-5951bdc3a89f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:19.024571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-49e47606-2829-4a1f-9313-5951bdc3a89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:19.119931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:19.271021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dbd87-8798-46eb-acf7-f35edf219a66 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:19.272536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d47dbd87-8798-46eb-acf7-f35edf219a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:19.418050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dbd87-8798-46eb-acf7-f35edf219a66 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:19.430356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:19.533190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:19.533808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:19.534093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:19.534387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:19.535997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:19.625351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:19.627116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe5548-55e0-4a3e-a60d-bc3e2f9ef19b service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:19.645379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 715/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:20.507358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9d57-b24a-4c90-b887-89496a3121d2 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:20.507838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9d57-b24a-4c90-b887-89496a3121d2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:20.507934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b9d57-b24a-4c90-b887-89496a3121d2 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:20.509349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:20.514822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204348f-7640-4663-9f9b-6fe298087318 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:20.515275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204348f-7640-4663-9f9b-6fe298087318 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:20.515411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204348f-7640-4663-9f9b-6fe298087318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:20.538226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:20.595086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2204348f-7640-4663-9f9b-6fe298087318 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:20.596303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2204348f-7640-4663-9f9b-6fe298087318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:20.690862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:20.842693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b9d57-b24a-4c90-b887-89496a3121d2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:20.844277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-215b9d57-b24a-4c90-b887-89496a3121d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:20.954049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:20.988731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b9d57-b24a-4c90-b887-89496a3121d2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:21.100797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:21.101334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:21.101616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:21.101886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:21.103559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:21.182912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:21.197109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:21.198781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f673a62-2d59-41f0-8b8a-cf9ed048fa18 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:21.218892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 709/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:22.083418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b56b1-cf13-45c7-87c0-cd2a57ea89f2 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:22.084011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b56b1-cf13-45c7-87c0-cd2a57ea89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:22.084011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b56b1-cf13-45c7-87c0-cd2a57ea89f2 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:22.085308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:22.090142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fb4f1-9d0a-4819-82cb-853b17db6459 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:22.090631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fb4f1-9d0a-4819-82cb-853b17db6459 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:22.090744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fb4f1-9d0a-4819-82cb-853b17db6459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:22.165131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-621fb4f1-9d0a-4819-82cb-853b17db6459 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:22.166366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-621fb4f1-9d0a-4819-82cb-853b17db6459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:22.264019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:22.417876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b56b1-cf13-45c7-87c0-cd2a57ea89f2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:22.419359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3d4b56b1-cf13-45c7-87c0-cd2a57ea89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:22.505270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:22.568079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b56b1-cf13-45c7-87c0-cd2a57ea89f2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:22.678213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:22.678771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:22.679050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:22.679318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:22.680896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:22.728167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:22.768983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:22.770615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7b66a-a13e-4e22-b401-e8925628ce5e service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:22.788628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 718/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:23.669537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d6d02-f624-4992-ac9c-9b6e172d2420 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:23.670158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d6d02-f624-4992-ac9c-9b6e172d2420 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:23.670158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d6d02-f624-4992-ac9c-9b6e172d2420 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:23.671553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:23.676379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014240-bfdd-4a82-a96b-ceb000cec54e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:23.676763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014240-bfdd-4a82-a96b-ceb000cec54e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:23.676929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014240-bfdd-4a82-a96b-ceb000cec54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:23.756167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0b014240-bfdd-4a82-a96b-ceb000cec54e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:23.757438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0b014240-bfdd-4a82-a96b-ceb000cec54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:23.855357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:24.004980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d6d02-f624-4992-ac9c-9b6e172d2420 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:24.006678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-415d6d02-f624-4992-ac9c-9b6e172d2420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:24.055818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:24.155035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d6d02-f624-4992-ac9c-9b6e172d2420 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:24.267701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:24.268243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:24.268529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:24.268800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:24.270444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:24.277491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:24.362323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:24.363967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e712a-e26c-4fbb-a8c6-e24449f065bb service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:24.382030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 712/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:25.246908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49afa7-19ed-4949-800c-8388c5ef57c3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:25.248074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49afa7-19ed-4949-800c-8388c5ef57c3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:25.248074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49afa7-19ed-4949-800c-8388c5ef57c3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:25.249390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:25.254179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0322d69-9a6d-409e-8948-3602091a012a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:25.254636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0322d69-9a6d-409e-8948-3602091a012a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:25.254765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0322d69-9a6d-409e-8948-3602091a012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:25.329217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c0322d69-9a6d-409e-8948-3602091a012a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:25.330442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c0322d69-9a6d-409e-8948-3602091a012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:25.396360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:25.424632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:25.577990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49afa7-19ed-4949-800c-8388c5ef57c3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:25.579547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4c49afa7-19ed-4949-800c-8388c5ef57c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:25.597608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:25.724320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49afa7-19ed-4949-800c-8388c5ef57c3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:25.811122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:25.840096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:25.840633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:25.841740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:25.842246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:25.844502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:25.937920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:25.939564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe404f-b3fe-45aa-974e-adce5a185d0b service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:25.957470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 721/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:26.831328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147e2e8-f991-4357-ad59-497b13a63590 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:26.831849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147e2e8-f991-4357-ad59-497b13a63590 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:26.831849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147e2e8-f991-4357-ad59-497b13a63590 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:26.833262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:26.838140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a9f6-8d1f-4681-a9df-1a45db48350f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:26.838626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a9f6-8d1f-4681-a9df-1a45db48350f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:26.838733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1a9f6-8d1f-4681-a9df-1a45db48350f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:26.916097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-56b1a9f6-8d1f-4681-a9df-1a45db48350f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:26.917394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-56b1a9f6-8d1f-4681-a9df-1a45db48350f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:26.959929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:27.016689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:27.174986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147e2e8-f991-4357-ad59-497b13a63590 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:27.176459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5147e2e8-f991-4357-ad59-497b13a63590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:27.321028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147e2e8-f991-4357-ad59-497b13a63590 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:27.373901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:27.432375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:27.433024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:27.433337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:27.433606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:27.435898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:27.523312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:27.524920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af3e6d8-5fd5-415d-bc41-c809bf356e20 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:27.542731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 715/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:28.387192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d1c5-9c39-4692-b217-217cdc712951 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:28.387794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d1c5-9c39-4692-b217-217cdc712951 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:28.387960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d1c5-9c39-4692-b217-217cdc712951 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:28.389333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:28.394286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0927-bfbf-4944-8cff-2bfd233719ef None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:28.394819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0927-bfbf-4944-8cff-2bfd233719ef None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:28.394926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc0927-bfbf-4944-8cff-2bfd233719ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:28.473249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-75cc0927-bfbf-4944-8cff-2bfd233719ef None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:28.474244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-75cc0927-bfbf-4944-8cff-2bfd233719ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:28.476295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:28.575812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:28.726853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d1c5-9c39-4692-b217-217cdc712951 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:28.728297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1c60d1c5-9c39-4692-b217-217cdc712951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:28.875449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d1c5-9c39-4692-b217-217cdc712951 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:28.902188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:28.989385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:28.989915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:28.990197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:28.990506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:28.992119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:29.082761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:29.084386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2beb59-365f-4950-8bc2-4140df788d9a service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:29.102308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 724/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:30.935499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27799231-0acd-4a0e-9d06-60a286c4c618 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:30.939797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27799231-0acd-4a0e-9d06-60a286c4c618 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:30.939797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27799231-0acd-4a0e-9d06-60a286c4c618 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:30.939797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:30.942849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7ffa0-fc15-4ecf-bfc4-52e573559b4b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:30.943243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7ffa0-fc15-4ecf-bfc4-52e573559b4b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:30.943399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7ffa0-fc15-4ecf-bfc4-52e573559b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:30.986357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:31.010808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a9c7ffa0-fc15-4ecf-bfc4-52e573559b4b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:31.012010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a9c7ffa0-fc15-4ecf-bfc4-52e573559b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:31.107673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:31.406818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27799231-0acd-4a0e-9d06-60a286c4c618 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:31.408384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-27799231-0acd-4a0e-9d06-60a286c4c618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:31.514587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27799231-0acd-4a0e-9d06-60a286c4c618 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:31.547088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:31.589703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:31.590313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:31.591121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:31.591409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:31.593025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:31.704038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:31.705416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8fed3-8542-46d2-b6fd-cd3f20cf33c7 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:31.724585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 718/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:33.790052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2899f-7d22-44b9-870b-1de505b88d46 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:33.790730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2899f-7d22-44b9-870b-1de505b88d46 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:33.790730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2899f-7d22-44b9-870b-1de505b88d46 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:33.792005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:33.797840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78a04e-3187-4e4c-aa0d-4f58c0289891 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:33.798326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78a04e-3187-4e4c-aa0d-4f58c0289891 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:33.798441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78a04e-3187-4e4c-aa0d-4f58c0289891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:33.867567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8a78a04e-3187-4e4c-aa0d-4f58c0289891 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:33.868808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8a78a04e-3187-4e4c-aa0d-4f58c0289891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:33.881553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:33.966650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:34.250843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2899f-7d22-44b9-870b-1de505b88d46 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:34.252448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4ef2899f-7d22-44b9-870b-1de505b88d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:34.353338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2899f-7d22-44b9-870b-1de505b88d46 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:34.365697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:34.426694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:34.427313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:34.427587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:34.427854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:34.429457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:34.539501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:34.541370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43e17b-9ebd-4add-9009-474677f526f0 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:34.560144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 727/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:36.829950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207136-b807-43ff-babd-2a4cb9bc3fce None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:36.830503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207136-b807-43ff-babd-2a4cb9bc3fce None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:36.830503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207136-b807-43ff-babd-2a4cb9bc3fce None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:36.831873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:36.837255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d9ba-9f94-4d9b-befd-acf1c16057e8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:36.837679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d9ba-9f94-4d9b-befd-acf1c16057e8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:36.837789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d9ba-9f94-4d9b-befd-acf1c16057e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:36.885474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:36.903950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1959d9ba-9f94-4d9b-befd-acf1c16057e8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:36.907777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1959d9ba-9f94-4d9b-befd-acf1c16057e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:37.002752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:37.290685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207136-b807-43ff-babd-2a4cb9bc3fce None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:37.292222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4c207136-b807-43ff-babd-2a4cb9bc3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:37.394244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207136-b807-43ff-babd-2a4cb9bc3fce None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:37.434132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:37.467167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:37.468316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:37.468316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:37.468316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:37.469905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:37.593308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:37.594718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b68119-a43d-4501-a68a-60a273029a92 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:37.614092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 721/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:39.652406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ade63-08c3-4851-a01b-5740f38e96b3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:39.653504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ade63-08c3-4851-a01b-5740f38e96b3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:39.653504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ade63-08c3-4851-a01b-5740f38e96b3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:39.654848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:39.660219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c760c2-c350-4bb8-828a-d73571cfd726 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:39.660660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c760c2-c350-4bb8-828a-d73571cfd726 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:39.660768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c760c2-c350-4bb8-828a-d73571cfd726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:39.726583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a1c760c2-c350-4bb8-828a-d73571cfd726 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:39.727803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a1c760c2-c350-4bb8-828a-d73571cfd726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:39.749203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:39.828366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:40.113133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ade63-08c3-4851-a01b-5740f38e96b3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:40.114723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1b6ade63-08c3-4851-a01b-5740f38e96b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:40.218611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ade63-08c3-4851-a01b-5740f38e96b3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:40.238961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:40.292835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:40.294019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:40.294317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:40.294620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:40.296213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:40.405024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:40.406427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5142db-2d13-451a-9655-8cf446fcd300 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:40.424701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 730/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:42.471434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f0c5-ec38-469b-a8fe-9e2a43d478c9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:42.472856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f0c5-ec38-469b-a8fe-9e2a43d478c9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:42.473144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f0c5-ec38-469b-a8fe-9e2a43d478c9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:42.474872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:42.481005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d02a63-027c-41d7-9dfe-f9a788a14c87 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:42.481532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d02a63-027c-41d7-9dfe-f9a788a14c87 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:42.482111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d02a63-027c-41d7-9dfe-f9a788a14c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:42.546996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:42.549646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-95d02a63-027c-41d7-9dfe-f9a788a14c87 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:42.550919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-95d02a63-027c-41d7-9dfe-f9a788a14c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:42.655670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:42.939561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061f0c5-ec38-469b-a8fe-9e2a43d478c9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:42.941147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e061f0c5-ec38-469b-a8fe-9e2a43d478c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:43.045933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061f0c5-ec38-469b-a8fe-9e2a43d478c9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:43.094234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:43.118358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:43.118973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:43.119261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:43.119576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:43.121160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:43.227882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:43.229327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d179c2-0271-4e89-a30c-0c0454ecadb2 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:43.247946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 724/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:45.287402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a468181-c800-420f-94f7-cdb23af79351 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:45.287955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a468181-c800-420f-94f7-cdb23af79351 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:45.287955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a468181-c800-420f-94f7-cdb23af79351 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:45.289304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:45.295367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bfba4-c5ff-45dc-8f0b-85acccacbb2b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:45.295817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bfba4-c5ff-45dc-8f0b-85acccacbb2b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:45.295916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bfba4-c5ff-45dc-8f0b-85acccacbb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:45.363143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cb0bfba4-c5ff-45dc-8f0b-85acccacbb2b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:45.364364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cb0bfba4-c5ff-45dc-8f0b-85acccacbb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:45.393644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:45.460270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:26:45.742474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a468181-c800-420f-94f7-cdb23af79351 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:45.744046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7a468181-c800-420f-94f7-cdb23af79351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:45.969825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a468181-c800-420f-94f7-cdb23af79351 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:45.997769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:46.043808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:46.044432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:46.044689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:46.044955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:46.046651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:46.160845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:46.162311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d830c-fc50-4c73-ac27-3ad6a28f67dc service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:46.182072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 733/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:48.184743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d46ef-c178-408c-925b-53ae7e13c8cf None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:48.185272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d46ef-c178-408c-925b-53ae7e13c8cf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:48.185272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d46ef-c178-408c-925b-53ae7e13c8cf None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:48.186644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:48.191804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378f96-542b-4f37-a940-cee71c14dd47 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:48.192236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378f96-542b-4f37-a940-cee71c14dd47 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:48.192349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b378f96-542b-4f37-a940-cee71c14dd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:48.257268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7b378f96-542b-4f37-a940-cee71c14dd47 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:48.258544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7b378f96-542b-4f37-a940-cee71c14dd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:48.267397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:48.355546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:48.639901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d46ef-c178-408c-925b-53ae7e13c8cf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:48.641486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bc5d46ef-c178-408c-925b-53ae7e13c8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:48.744981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d46ef-c178-408c-925b-53ae7e13c8cf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:48.799804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:48.818662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:48.821249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:48.821249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:48.821249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:48.821532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:48.930259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:48.936212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec8f7d-9093-4dfd-b7df-c89e79886aec service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:48.951630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 727/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:50.927652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08408e6b-4eee-435c-8567-16685d2fe0d2 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:50.928197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08408e6b-4eee-435c-8567-16685d2fe0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:50.928197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08408e6b-4eee-435c-8567-16685d2fe0d2 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:50.929548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:50.935406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3447-a56f-47a9-af68-627364c8c206 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:50.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3447-a56f-47a9-af68-627364c8c206 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:50.937746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3447-a56f-47a9-af68-627364c8c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:51.004275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-319f3447-a56f-47a9-af68-627364c8c206 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:51.005512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-319f3447-a56f-47a9-af68-627364c8c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:51.044423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:51.102032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:51.392210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08408e6b-4eee-435c-8567-16685d2fe0d2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:51.393746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-08408e6b-4eee-435c-8567-16685d2fe0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:51.495710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08408e6b-4eee-435c-8567-16685d2fe0d2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:51.534598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:51.570429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:51.571029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:51.571306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:51.571571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:51.573147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:51.682089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:51.683521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb16600-03f2-4142-ab2d-43124cef1e49 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:51.701666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 736/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:53.651206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad77e6-4f4e-4a64-8ebc-0fbe6efc2f8d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:53.651806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad77e6-4f4e-4a64-8ebc-0fbe6efc2f8d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:53.651806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad77e6-4f4e-4a64-8ebc-0fbe6efc2f8d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:53.653911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:53.659337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6465a-6084-4afe-a207-a4ecc0856315 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:53.659775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6465a-6084-4afe-a207-a4ecc0856315 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:53.659879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6465a-6084-4afe-a207-a4ecc0856315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:53.725897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-eca6465a-6084-4afe-a207-a4ecc0856315 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:53.731314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-eca6465a-6084-4afe-a207-a4ecc0856315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:53.747370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:53.828492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:54.110043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad77e6-4f4e-4a64-8ebc-0fbe6efc2f8d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:54.111636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9cad77e6-4f4e-4a64-8ebc-0fbe6efc2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:54.214051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad77e6-4f4e-4a64-8ebc-0fbe6efc2f8d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:54.280037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:54.286696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:54.287306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:54.287588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:54.287861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:54.289452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:54.397554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:54.398970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c332f9-b5de-4b04-8ef5-b9d3887899b4 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:54.417096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 730/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:26:56.315510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52384064-afa1-4d4e-a7c9-65b252767cf3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:56.316084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52384064-afa1-4d4e-a7c9-65b252767cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:56.316084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52384064-afa1-4d4e-a7c9-65b252767cf3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:56.317391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:26:56.322757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2b070-1ecd-4e65-aa03-7a157787ea34 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:56.323171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2b070-1ecd-4e65-aa03-7a157787ea34 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:56.323324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2b070-1ecd-4e65-aa03-7a157787ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:56.388926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-add2b070-1ecd-4e65-aa03-7a157787ea34 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:26:56.390143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-add2b070-1ecd-4e65-aa03-7a157787ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:56.438711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:56.487429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:26:56.776121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52384064-afa1-4d4e-a7c9-65b252767cf3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:26:56.777949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-52384064-afa1-4d4e-a7c9-65b252767cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:56.879398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52384064-afa1-4d4e-a7c9-65b252767cf3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:26:56.933991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:26:56.960111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:26:56.960720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:26:56.960998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:26:56.961265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:26:56.962926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:26:57.071636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:26:57.072992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684fa6-c48c-44ba-8f14-4fa4ed3b01aa service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:26:57.091876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 739/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:06.485691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=22 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:07.089144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=66 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:17.048075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66619f9-1cde-4242-85ac-7e2eac083a61 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:17.048643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66619f9-1cde-4242-85ac-7e2eac083a61 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:17.048739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66619f9-1cde-4242-85ac-7e2eac083a61 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:17.050086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 732/1471] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:17.055295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e746e5-33af-4b42-8bff-b69e47ecb395 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:17.055693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e746e5-33af-4b42-8bff-b69e47ecb395 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:17.055794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e746e5-33af-4b42-8bff-b69e47ecb395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:17.122584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e5e746e5-33af-4b42-8bff-b69e47ecb395 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:17.123813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e5e746e5-33af-4b42-8bff-b69e47ecb395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:17.220352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 740/1472] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:27:17.517608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320074d8-0c77-4fc0-8729-5451a2fd5a0f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:17.517921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320074d8-0c77-4fc0-8729-5451a2fd5a0f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:17.518196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320074d8-0c77-4fc0-8729-5451a2fd5a0f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:17.519588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 733/1473] 10.222.0.13 () {54 vars in 961 bytes} [Sat Sep 3 01:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:17.524427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531b4b-40b7-4389-88ff-c317931f8426 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:17.524719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531b4b-40b7-4389-88ff-c317931f8426 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:17.524995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531b4b-40b7-4389-88ff-c317931f8426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:17.599547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8c531b4b-40b7-4389-88ff-c317931f8426 None None] MFA Rules not processed for user `cbc9f7e49e5946f48df125d4efd80dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:17.600747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8c531b4b-40b7-4389-88ff-c317931f8426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:17.621050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:17.676211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 741/1474] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Sep 3 01:27:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:17.683752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320074d8-0c77-4fc0-8729-5451a2fd5a0f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:17.685931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-320074d8-0c77-4fc0-8729-5451a2fd5a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:17.789816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320074d8-0c77-4fc0-8729-5451a2fd5a0f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:17.864145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:17.864760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:17.865038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:17.865308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:17.866942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:17.920978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:17.977435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:17.978925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ec0fc-6b02-40f8-b8d0-8011a4b8f1d8 service glance] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:17.997958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 734/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:20.000583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e89505-2024-4135-a320-e60e86f9010e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:20.001147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e89505-2024-4135-a320-e60e86f9010e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:20.001147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e89505-2024-4135-a320-e60e86f9010e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:20.002512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 742/1476] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:20.007748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f342-62fc-4cd6-94c9-b06a7d3e32a9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:20.008151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f342-62fc-4cd6-94c9-b06a7d3e32a9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:20.008429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104f342-62fc-4cd6-94c9-b06a7d3e32a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:20.074478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0104f342-62fc-4cd6-94c9-b06a7d3e32a9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:20.075969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0104f342-62fc-4cd6-94c9-b06a7d3e32a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:20.172290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 735/1477] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:20.444051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e89505-2024-4135-a320-e60e86f9010e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:20.445521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-95e89505-2024-4135-a320-e60e86f9010e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:20.571093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e89505-2024-4135-a320-e60e86f9010e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:20.619087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:20.663102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:20.663708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:20.663989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:20.664259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:20.664894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:20.666714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:27:20.666714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:27:20.666714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:27:20.667845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9040-cc3f-4824-9623-48748416568a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:20.687722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 743/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Sep 3 01:27:20 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:27:22.626138 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721e29c-4a58-4f22-996f-e5887a46ef72 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:22.626747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721e29c-4a58-4f22-996f-e5887a46ef72 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:22.626747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4721e29c-4a58-4f22-996f-e5887a46ef72 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:22.628100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 736/1479] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:22.633456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e674-4a09-4129-b1c3-8067827437d5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:22.633864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e674-4a09-4129-b1c3-8067827437d5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:22.634060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472e674-4a09-4129-b1c3-8067827437d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:22.703702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5472e674-4a09-4129-b1c3-8067827437d5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:22.704965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5472e674-4a09-4129-b1c3-8067827437d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:22.803849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 744/1480] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:23.099575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721e29c-4a58-4f22-996f-e5887a46ef72 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:23.101174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4721e29c-4a58-4f22-996f-e5887a46ef72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:23.111466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:23.246640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4721e29c-4a58-4f22-996f-e5887a46ef72 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:23.325038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:23.358306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:23.358876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:23.359157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:23.359426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:23.361034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:23.448503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:23.450187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ce658-1ba5-4b32-aa8c-704b636b2b47 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:23.468353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 737/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:25.588389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e0bd-8e59-461e-8357-0e7471f45f19 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:25.589001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e0bd-8e59-461e-8357-0e7471f45f19 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:25.589001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e0bd-8e59-461e-8357-0e7471f45f19 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:25.590483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 745/1482] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:25.596048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2999b70-526e-4180-b03d-978922c524f3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:25.596501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2999b70-526e-4180-b03d-978922c524f3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:25.596610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2999b70-526e-4180-b03d-978922c524f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:25.662657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2999b70-526e-4180-b03d-978922c524f3 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:25.663873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2999b70-526e-4180-b03d-978922c524f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:25.709768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:25.760223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 738/1483] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:26.049226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e0bd-8e59-461e-8357-0e7471f45f19 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:26.050827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e626e0bd-8e59-461e-8357-0e7471f45f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:26.074102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:26.154939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e0bd-8e59-461e-8357-0e7471f45f19 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:26.227985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:26.228587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:26.228861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:26.229126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:26.230943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:26.303795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:26.339277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:26.340755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ec1bb-2464-4790-b9a1-d4c0b38bb92f service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:26.358438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 746/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:28.314409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aa1a4-ca29-4a42-a415-730a6b45f1bd None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:28.314937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aa1a4-ca29-4a42-a415-730a6b45f1bd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:28.314937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aa1a4-ca29-4a42-a415-730a6b45f1bd None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:28.316231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 739/1485] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:28.321322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900528e-950a-4e01-b632-80a77a57d003 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:28.321768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900528e-950a-4e01-b632-80a77a57d003 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:28.321872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900528e-950a-4e01-b632-80a77a57d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:28.388752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3900528e-950a-4e01-b632-80a77a57d003 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:28.389952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3900528e-950a-4e01-b632-80a77a57d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:28.483701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 747/1486] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:28.775221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1aa1a4-ca29-4a42-a415-730a6b45f1bd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:28.776772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e1aa1a4-ca29-4a42-a415-730a6b45f1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:28.887375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1aa1a4-ca29-4a42-a415-730a6b45f1bd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:28.959314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:28.961101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:28.961731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:28.962013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:28.962279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:28.963919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:29.071726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:29.073097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae2f1f-a7ea-40c3-8407-e5ece2bafafb service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:29.091135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 740/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:31.178724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213163d-2c72-4bce-abcd-f4dc55c00bbf None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:31.179253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213163d-2c72-4bce-abcd-f4dc55c00bbf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:31.179253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213163d-2c72-4bce-abcd-f4dc55c00bbf None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:31.180563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 748/1488] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:31.185725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb8369-84dd-4b47-85b8-e15022f415b2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:31.186106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb8369-84dd-4b47-85b8-e15022f415b2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:31.186292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb8369-84dd-4b47-85b8-e15022f415b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:31.251712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-90cb8369-84dd-4b47-85b8-e15022f415b2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:31.252935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-90cb8369-84dd-4b47-85b8-e15022f415b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:31.305169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:31.348818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 741/1489] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:31.641821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213163d-2c72-4bce-abcd-f4dc55c00bbf None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:31.643536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3213163d-2c72-4bce-abcd-f4dc55c00bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:31.680206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:31.749414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213163d-2c72-4bce-abcd-f4dc55c00bbf None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:31.822937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:31.823548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:31.823824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:31.824090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:31.826199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:31.908729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:31.937139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:31.938634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19222573-2b2c-4bf5-beff-b654711f63cf service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:31.957175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 749/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:34.279954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bf7fa-81d8-40ea-8438-1e062172bb60 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:34.280537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bf7fa-81d8-40ea-8438-1e062172bb60 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:34.280537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bf7fa-81d8-40ea-8438-1e062172bb60 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:34.281854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 742/1491] 10.222.0.13 () {54 vars in 927 bytes} [Sat Sep 3 01:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:27:34.287047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a864268-d002-4b0e-9d3b-eb103b18e320 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:34.287455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a864268-d002-4b0e-9d3b-eb103b18e320 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:34.287602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a864268-d002-4b0e-9d3b-eb103b18e320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:34.354085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2a864268-d002-4b0e-9d3b-eb103b18e320 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:27:34.355931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2a864268-d002-4b0e-9d3b-eb103b18e320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:34.450810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 750/1492] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Sep 3 01:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:27:34.741814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bf7fa-81d8-40ea-8438-1e062172bb60 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:27:34.743392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5b9bf7fa-81d8-40ea-8438-1e062172bb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:34.848364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bf7fa-81d8-40ea-8438-1e062172bb60 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:27:34.921509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:27:34.922119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:27:34.922429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:27:34.922702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:27:34.924320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:27:34.935320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:35.151501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:27:35.153763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c914-f5e0-4ac1-a375-6868bea1dfaa service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:27:35.171596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 743/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:27:44.448940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:27:45.169431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=50 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:19.116521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:19.117131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:19.117131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:19.193152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:29:19.194378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:19.287680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 751/1494] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:29:19.296156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:19.296670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:19.296872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:19.371118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:29:19.372774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:19.466526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 744/1495] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:29:19.471195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:19.472448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:19.484829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daf8ef-dd62-40dc-9a0d-d3f820e10001 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:19.545448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:19.580083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:19.580682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:19.580956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:19.581218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:19.581853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:29:19.582149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:19.583189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:19.583189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:29:19.583189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:29:19.584061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:19.595158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 752/1496] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:29:19.603651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:19.613797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9733dab-2399-4143-b19c-d0b20e69bf8d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:19.674919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:19.707844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:19.708507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:19.708973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:19.709280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:19.710536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:19.712298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:19.712298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:29:19.712298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:29:19.713487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:19.854788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 745/1497] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:29:19 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:29:19.859587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:19.861106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:19.894108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae2e54-d7d6-4fc4-b1c1-1111734114fa admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:19.988514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:19.989040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:19.989323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:19.989594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:19.990229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:19.991285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:19.991285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:29:19.991285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:29:19.992497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:20.075095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 753/1498] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:29:19 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:29:20.079429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:20.080970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:20.140766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1d5f2-dd75-4fd2-8466-88cc362a4233 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:20.222438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:20.233302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:20.233907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:20.234184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:20.234485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:20.235120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:20.236140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:20.236140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:29:20.236140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:29:20.237007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:20.249360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 746/1499] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:29:20.253120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:20.254790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:20.266602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f5fa5-93c2-4746-9a36-5d6f1684f9ba admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:20.292029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:20.362483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:20.363086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:20.363363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:20.363626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:20.364257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:20.365258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:20.365258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:29:20.365258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:29:20.366137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fca04-b422-4222-8fe6-8868eb2a73dd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:20.378711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 754/1500] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:29:20.382910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:20.384420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:20.417589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3faaa-9d76-408b-a566-20c02f70b7f2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:20.508173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:20.508756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:20.509032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:20.509300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] PATH_INFO: `/v3/projects/5cb8e8dcac3d443b86a815a9c29969f8/users/9a26de3450a84dd49c6070abc66ce5b1/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:20.532074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb8e8dcac3d443b86a815a9c29969f8, user_id=9a26de3450a84dd49c6070abc66ce5b1, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:20.533113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:20.533113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:29:20.533113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:29:20.535102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b230c9-e447-4a34-922d-e34ee9f45adb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:20.610003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 747/1501] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:29:20 2022] PUT /identity/v3/projects/5cb8e8dcac3d443b86a815a9c29969f8/users/9a26de3450a84dd49c6070abc66ce5b1/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:29:20.622067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373accce-e0cd-4129-9c1d-dd468f1be3aa None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:20.622469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373accce-e0cd-4129-9c1d-dd468f1be3aa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:20.622751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373accce-e0cd-4129-9c1d-dd468f1be3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:20.679327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-373accce-e0cd-4129-9c1d-dd468f1be3aa None None] MFA Rules not processed for user `9a26de3450a84dd49c6070abc66ce5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:29:20.680589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-373accce-e0cd-4129-9c1d-dd468f1be3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:20.757403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 755/1502] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:29:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:29:20.772259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c275b4-9d4e-41a6-835e-fbf8c69c8aee None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:20.772548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c275b4-9d4e-41a6-835e-fbf8c69c8aee None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:20.772823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c275b4-9d4e-41a6-835e-fbf8c69c8aee None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:20.774163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 748/1503] 10.222.0.13 () {54 vars in 964 bytes} [Sat Sep 3 01:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:29:20.779757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4af7f-1679-47d6-8791-68d08df404b3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:20.780045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4af7f-1679-47d6-8791-68d08df404b3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:20.780319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4af7f-1679-47d6-8791-68d08df404b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:20.811629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:20.858495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-50c4af7f-1679-47d6-8791-68d08df404b3 None None] MFA Rules not processed for user `edb970ecb499474f87e02b9ee49c8c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:29:20.859702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-50c4af7f-1679-47d6-8791-68d08df404b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:20.935758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 756/1504] 10.222.0.13 () {58 vars in 1061 bytes} [Sat Sep 3 01:29:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:29:20.943505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c275b4-9d4e-41a6-835e-fbf8c69c8aee None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:20.945117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-68c275b4-9d4e-41a6-835e-fbf8c69c8aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:20.995127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:21.049574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c275b4-9d4e-41a6-835e-fbf8c69c8aee None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:21.122461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:21.123069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:21.123348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:21.123617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:21.125207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:21.222409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:21.223856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:21.227331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:21.241961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 749/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:29:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:29:21.288030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362027b9-4e2b-4158-a32d-08c88d5b83d1 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:21.288462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362027b9-4e2b-4158-a32d-08c88d5b83d1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:21.288569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362027b9-4e2b-4158-a32d-08c88d5b83d1 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:21.289842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 757/1506] 10.222.0.13 () {52 vars in 830 bytes} [Sat Sep 3 01:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:29:21.293410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:21.295316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:21.429388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce124b-2446-4e17-af9a-c92ae9bb7d68 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:21.523155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:21.523790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:21.524071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:21.524343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] PATH_INFO: `/v3/users/9a26de3450a84dd49c6070abc66ce5b1` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:21.537653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a26de3450a84dd49c6070abc66ce5b1)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:21.538733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:21.538733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:29:21.538733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:29:21.539945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:21.550498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:21.765365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a26de3450a84dd49c6070abc66ce5b1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:21.827211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a26de3450a84dd49c6070abc66ce5b1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:21.827848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a26de3450a84dd49c6070abc66ce5b1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:21.851726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a26de3450a84dd49c6070abc66ce5b1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:21.866287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a26de3450a84dd49c6070abc66ce5b1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:21.872626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 750/1507] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:29:21 2022] DELETE /identity/v3/users/9a26de3450a84dd49c6070abc66ce5b1 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:29:21.877284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:21.877575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:21.877850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:21.935937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:29:21.937153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:22.015581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:22.027839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 758/1508] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:29:22.038417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:22.042883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:22.142284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ffcdb-6457-4ef1-a687-b297549e70de admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:22.196870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:22.215994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:22.216619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:22.216902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:22.217170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:22.218810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:22.329686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:22.331147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a031384-0ad2-4196-95cd-32e2be060ca9 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:22.349745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 751/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:29:22.830091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:29:22.830672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:29:22.842724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966f338-4c60-44a6-897f-080e7edca971 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:29:22.939301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:29:22.939851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:29:22.940128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:29:22.940394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] PATH_INFO: `/v3/projects/5cb8e8dcac3d443b86a815a9c29969f8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:29:22.950884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb8e8dcac3d443b86a815a9c29969f8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:29:22.952022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:29:22.952022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:29:22.952022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:29:22.953229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:29:23.062570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb8e8dcac3d443b86a815a9c29969f8', 'request_id': 'req-6efead96-7042-4326-ac86-1237f0343e40'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:23.138396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb8e8dcac3d443b86a815a9c29969f8', 'request_id': 'req-6efead96-7042-4326-ac86-1237f0343e40'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:23.145201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb8e8dcac3d443b86a815a9c29969f8', 'request_id': 'req-6efead96-7042-4326-ac86-1237f0343e40'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:23.176856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] The token cache is being invalidate because project 5cb8e8dcac3d443b86a815a9c29969f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:29:23.177147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:23.177577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6efead96-7042-4326-ac86-1237f0343e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:29:23.179036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 759/1510] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:29:22 2022] DELETE /identity/v3/projects/5cb8e8dcac3d443b86a815a9c29969f8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:29:32.347124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=62 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:29:33.175036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:30:52.522861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd40a-3fe8-46a4-9791-4ba352bb79b5 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:52.523508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd40a-3fe8-46a4-9791-4ba352bb79b5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:52.523508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdd40a-3fe8-46a4-9791-4ba352bb79b5 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:52.524807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:30:52.530121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9a766-5068-4282-b1b5-6682163e47fe None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:52.530585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9a766-5068-4282-b1b5-6682163e47fe None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:52.530768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9a766-5068-4282-b1b5-6682163e47fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:52.616889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-07f9a766-5068-4282-b1b5-6682163e47fe None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:30:52.618112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-07f9a766-5068-4282-b1b5-6682163e47fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:30:52.735641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:30:52.892318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd40a-3fe8-46a4-9791-4ba352bb79b5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:30:52.894849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7cbdd40a-3fe8-46a4-9791-4ba352bb79b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:30:53.023406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbdd40a-3fe8-46a4-9791-4ba352bb79b5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:30:53.029369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d8efb-434a-49e8-975a-943ef5e2ac40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:30:53.029988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d8efb-434a-49e8-975a-943ef5e2ac40 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:53.030268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d8efb-434a-49e8-975a-943ef5e2ac40 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:53.030628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d8efb-434a-49e8-975a-943ef5e2ac40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:53.039364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d8efb-434a-49e8-975a-943ef5e2ac40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:30:53.040978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d8efb-434a-49e8-975a-943ef5e2ac40 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:30:53.052142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 753/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:30:53.925670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db35233-0acd-493c-abec-d7332ffa74a9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:53.925670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db35233-0acd-493c-abec-d7332ffa74a9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:53.925670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db35233-0acd-493c-abec-d7332ffa74a9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:53.926653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:30:53.931757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eb5b5-50b6-4730-8332-aeb062527348 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:53.932165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eb5b5-50b6-4730-8332-aeb062527348 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:53.932310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302eb5b5-50b6-4730-8332-aeb062527348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:53.957617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-302eb5b5-50b6-4730-8332-aeb062527348 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:30:53.959292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-302eb5b5-50b6-4730-8332-aeb062527348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:30:53.967467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:30:54.122186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db35233-0acd-493c-abec-d7332ffa74a9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:30:54.137761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db35233-0acd-493c-abec-d7332ffa74a9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:30:54.144437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed46487-9d5e-4522-9d7c-c64131475d01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:30:54.145220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed46487-9d5e-4522-9d7c-c64131475d01 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:54.145502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed46487-9d5e-4522-9d7c-c64131475d01 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:54.145773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed46487-9d5e-4522-9d7c-c64131475d01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:54.154812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed46487-9d5e-4522-9d7c-c64131475d01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:30:54.157028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed46487-9d5e-4522-9d7c-c64131475d01 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:30:54.168542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 762/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:30:54.682062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1ba94-5eba-4e08-9bab-efc7a4d5aedb None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:54.682463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1ba94-5eba-4e08-9bab-efc7a4d5aedb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:54.683054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1ba94-5eba-4e08-9bab-efc7a4d5aedb None None] PATH_INFO: `/v3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:54.684762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 755/1517] 10.222.0.42 () {54 vars in 925 bytes} [Sat Sep 3 01:30:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:30:54.694380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80cae-af73-457b-b7b2-5053c9e0d08d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:54.694380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80cae-af73-457b-b7b2-5053c9e0d08d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:54.694380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a80cae-af73-457b-b7b2-5053c9e0d08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:54.757347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-00a80cae-af73-457b-b7b2-5053c9e0d08d None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:30:54.758619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-00a80cae-af73-457b-b7b2-5053c9e0d08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:30:54.778437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:30:54.840557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 763/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 3 01:30:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:30:59.037394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddee6b-7c99-4ded-b12f-2bd00783277d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:59.038023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddee6b-7c99-4ded-b12f-2bd00783277d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:59.038023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eddee6b-7c99-4ded-b12f-2bd00783277d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:59.039307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 756/1519] 10.222.0.13 () {54 vars in 961 bytes} [Sat Sep 3 01:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:30:59.044463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d8e6f-a5b2-4434-9247-355ed5f3c5c8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:59.044911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d8e6f-a5b2-4434-9247-355ed5f3c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:59.045020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d8e6f-a5b2-4434-9247-355ed5f3c5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:59.068201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-257d8e6f-a5b2-4434-9247-355ed5f3c5c8 None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:30:59.069402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-257d8e6f-a5b2-4434-9247-355ed5f3c5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:30:59.077971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 764/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Sep 3 01:30:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:30:59.085723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddee6b-7c99-4ded-b12f-2bd00783277d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:30:59.098976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eddee6b-7c99-4ded-b12f-2bd00783277d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:30:59.103829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daada02c-2a87-4898-ae77-6b6a2cc312c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:30:59.104417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daada02c-2a87-4898-ae77-6b6a2cc312c1 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:30:59.104692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daada02c-2a87-4898-ae77-6b6a2cc312c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:30:59.104961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daada02c-2a87-4898-ae77-6b6a2cc312c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:30:59.113536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daada02c-2a87-4898-ae77-6b6a2cc312c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:30:59.115146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daada02c-2a87-4898-ae77-6b6a2cc312c1 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:30:59.124559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 757/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:30:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:05.412733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8496c-8977-44d6-b3f4-59ffea216cdc None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:05.413305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8496c-8977-44d6-b3f4-59ffea216cdc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:05.413305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8496c-8977-44d6-b3f4-59ffea216cdc None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:05.414670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:31:05.419569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80940bb0-7701-4ba6-b5f0-864886f24557 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:05.420030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80940bb0-7701-4ba6-b5f0-864886f24557 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:05.420133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80940bb0-7701-4ba6-b5f0-864886f24557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:05.440245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-80940bb0-7701-4ba6-b5f0-864886f24557 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:31:05.441439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-80940bb0-7701-4ba6-b5f0-864886f24557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:05.449171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:31:05.604397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8496c-8977-44d6-b3f4-59ffea216cdc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:31:05.618704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8496c-8977-44d6-b3f4-59ffea216cdc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:31:05.624158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ca991-fd7f-420f-899c-1fc46b72f449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:31:05.624755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca991-fd7f-420f-899c-1fc46b72f449 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:05.625038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca991-fd7f-420f-899c-1fc46b72f449 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:05.625312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ca991-fd7f-420f-899c-1fc46b72f449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:05.633475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ca991-fd7f-420f-899c-1fc46b72f449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:31:05.637662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ca991-fd7f-420f-899c-1fc46b72f449 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:31:05.645005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 766/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:06.471764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd57aff-172d-4790-a5b0-035db2811fcf None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:06.472360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd57aff-172d-4790-a5b0-035db2811fcf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:06.472360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd57aff-172d-4790-a5b0-035db2811fcf None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:06.473656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 3 01:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:31:06.478423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf937f-5ad6-4492-a256-587420fff8ea None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:06.478870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf937f-5ad6-4492-a256-587420fff8ea None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:06.478979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf937f-5ad6-4492-a256-587420fff8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:06.498618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-26bf937f-5ad6-4492-a256-587420fff8ea None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:31:06.499817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-26bf937f-5ad6-4492-a256-587420fff8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:06.510982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 3 01:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:31:06.664114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd57aff-172d-4790-a5b0-035db2811fcf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:31:06.681069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd57aff-172d-4790-a5b0-035db2811fcf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:31:06.686690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac3b4b-7ab8-4300-8bc7-6894d7cca16b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:31:06.687302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac3b4b-7ab8-4300-8bc7-6894d7cca16b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:06.687583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac3b4b-7ab8-4300-8bc7-6894d7cca16b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:06.687859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac3b4b-7ab8-4300-8bc7-6894d7cca16b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:06.696570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac3b4b-7ab8-4300-8bc7-6894d7cca16b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:31:06.698177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac3b4b-7ab8-4300-8bc7-6894d7cca16b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:31:06.708028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 760/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:10.496037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9f525-7c3b-4be8-9d4f-9c5ab31b035a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:10.496545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9f525-7c3b-4be8-9d4f-9c5ab31b035a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:10.496545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9f525-7c3b-4be8-9d4f-9c5ab31b035a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:10.497887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:31:10.503932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97275539-9e56-41c3-a116-33a12f31fa5d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:10.504399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97275539-9e56-41c3-a116-33a12f31fa5d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:10.504510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97275539-9e56-41c3-a116-33a12f31fa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:10.526405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-97275539-9e56-41c3-a116-33a12f31fa5d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:31:10.527657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-97275539-9e56-41c3-a116-33a12f31fa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:10.537808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:31:10.829521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9f525-7c3b-4be8-9d4f-9c5ab31b035a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:31:10.832011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ccf9f525-7c3b-4be8-9d4f-9c5ab31b035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:10.912180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9f525-7c3b-4be8-9d4f-9c5ab31b035a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:31:10.915751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc535a5a-d0a3-4a3a-baff-841b547768c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:31:10.916360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc535a5a-d0a3-4a3a-baff-841b547768c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:10.916639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc535a5a-d0a3-4a3a-baff-841b547768c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:10.916909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc535a5a-d0a3-4a3a-baff-841b547768c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:10.926005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc535a5a-d0a3-4a3a-baff-841b547768c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:31:10.927523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc535a5a-d0a3-4a3a-baff-841b547768c3 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:31:10.936895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 769/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:12.843380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8cc03-a04f-4064-9598-69b917babc86 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:12.843920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8cc03-a04f-4064-9598-69b917babc86 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:12.843920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8cc03-a04f-4064-9598-69b917babc86 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:12.845295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:31:12.850650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05718880-da45-4406-8bc3-9e5e4db449cd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:12.851094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05718880-da45-4406-8bc3-9e5e4db449cd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:12.851208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05718880-da45-4406-8bc3-9e5e4db449cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:12.871685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-05718880-da45-4406-8bc3-9e5e4db449cd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:31:12.872885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05718880-da45-4406-8bc3-9e5e4db449cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:12.881412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:31:13.169510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8cc03-a04f-4064-9598-69b917babc86 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:31:13.174834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0ad8cc03-a04f-4064-9598-69b917babc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:13.188407 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8cc03-a04f-4064-9598-69b917babc86 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:31:13.192698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969609df-f77e-46ba-b27d-4f3bfd70de6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:31:13.193303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969609df-f77e-46ba-b27d-4f3bfd70de6d service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:13.193584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969609df-f77e-46ba-b27d-4f3bfd70de6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:13.193859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969609df-f77e-46ba-b27d-4f3bfd70de6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:13.202946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969609df-f77e-46ba-b27d-4f3bfd70de6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:31:13.204390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969609df-f77e-46ba-b27d-4f3bfd70de6d service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:31:13.214599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 763/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:16.734823 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3124619-5c25-4294-9642-8330f8f927c7 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:16.735390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3124619-5c25-4294-9642-8330f8f927c7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:16.735390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3124619-5c25-4294-9642-8330f8f927c7 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:16.736717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:31:16.742096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b502547-136d-46f2-aeb7-e5ed86fb66e0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:16.742520 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b502547-136d-46f2-aeb7-e5ed86fb66e0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:16.742671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b502547-136d-46f2-aeb7-e5ed86fb66e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:16.763178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2b502547-136d-46f2-aeb7-e5ed86fb66e0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:31:16.764380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2b502547-136d-46f2-aeb7-e5ed86fb66e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:16.772943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:31:17.328992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3124619-5c25-4294-9642-8330f8f927c7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:31:17.330741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c3124619-5c25-4294-9642-8330f8f927c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:17.394974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:31:17.408086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3124619-5c25-4294-9642-8330f8f927c7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:31:17.412209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb7259-5db3-429c-993d-a6586f177776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:31:17.412822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7259-5db3-429c-993d-a6586f177776 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:17.413101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7259-5db3-429c-993d-a6586f177776 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:17.413371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb7259-5db3-429c-993d-a6586f177776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:17.422113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb7259-5db3-429c-993d-a6586f177776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:31:17.423559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb7259-5db3-429c-993d-a6586f177776 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:31:17.433832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 772/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:19.450504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412e1c3-2840-4c86-88ef-88a3c909f98b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:19.450947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412e1c3-2840-4c86-88ef-88a3c909f98b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:19.451057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412e1c3-2840-4c86-88ef-88a3c909f98b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:19.452835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 3 01:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:31:19.458359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3749f-bfbf-408f-9985-5aa0354c4ffd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:19.458854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3749f-bfbf-408f-9985-5aa0354c4ffd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:19.459037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3749f-bfbf-408f-9985-5aa0354c4ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:19.480666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6ea3749f-bfbf-408f-9985-5aa0354c4ffd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:31:19.481898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6ea3749f-bfbf-408f-9985-5aa0354c4ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:19.492081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 3 01:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:31:20.070656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412e1c3-2840-4c86-88ef-88a3c909f98b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:31:20.072361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c412e1c3-2840-4c86-88ef-88a3c909f98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:31:20.085675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412e1c3-2840-4c86-88ef-88a3c909f98b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:31:20.089208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730bb40-8577-4d24-8d6d-02e327d6cf8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:31:20.089799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730bb40-8577-4d24-8d6d-02e327d6cf8b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:31:20.090074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730bb40-8577-4d24-8d6d-02e327d6cf8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:31:20.090365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730bb40-8577-4d24-8d6d-02e327d6cf8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:31:20.099677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730bb40-8577-4d24-8d6d-02e327d6cf8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:31:20.101095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730bb40-8577-4d24-8d6d-02e327d6cf8b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:31:20.110315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 766/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:31:29.474730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=11 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:31:30.106246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:47.135923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c113fd4-15cd-47ff-80d6-aab389eeb992 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.136581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c113fd4-15cd-47ff-80d6-aab389eeb992 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.136581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c113fd4-15cd-47ff-80d6-aab389eeb992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.157145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6d253-1c2b-4f3d-9beb-b7018ed2abf2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.157777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6d253-1c2b-4f3d-9beb-b7018ed2abf2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.157777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d6d253-1c2b-4f3d-9beb-b7018ed2abf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.167427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0c113fd4-15cd-47ff-80d6-aab389eeb992 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.168775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0c113fd4-15cd-47ff-80d6-aab389eeb992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.188968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 774/1540] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:32:47.193672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-46d6d253-1c2b-4f3d-9beb-b7018ed2abf2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.194977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-46d6d253-1c2b-4f3d-9beb-b7018ed2abf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.197361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.197901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.198406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.209051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 767/1541] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.218360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.218934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.219948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.229015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.230371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.247712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.249478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.253894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 775/1542] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:32:47.255993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.269940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 768/1543] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:32:47.272816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f0f1f-da32-491d-9270-c4b70f894528 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.274659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.282212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.283232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.283738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.284226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.285402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:47.285930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.289786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.293162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4b47-2d84-44a3-9407-6893be8effcb None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.310988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.311631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.311913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.312185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.312852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:47.313157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.314198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 776/1544] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:47.314332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:32:47.314332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:32:47.314332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:32:47.319575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.322016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.336238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b4cd9-7185-40f3-8485-b5919462ad4c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.338526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 769/1545] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:32:47.343037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.348608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.349626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.350142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.350663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.351557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.352843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:32:47.352843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:32:47.352843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:32:47.359327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e72bf-2803-4903-b711-38f41a0156eb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.359855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d8f6a-21e7-4e94-abcc-06ef03b75cdf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.370605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a968a67-f92c-49fb-8dc4-8f88841107eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.371250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968a67-f92c-49fb-8dc4-8f88841107eb admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.371535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968a67-f92c-49fb-8dc4-8f88841107eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.371811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a968a67-f92c-49fb-8dc4-8f88841107eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.378362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968a67-f92c-49fb-8dc4-8f88841107eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.379309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a968a67-f92c-49fb-8dc4-8f88841107eb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.485201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 770/1546] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:32:47.486581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2f368-2e4b-4bdb-8dee-547b0e9a4ec9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.486890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2f368-2e4b-4bdb-8dee-547b0e9a4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.487155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2f368-2e4b-4bdb-8dee-547b0e9a4ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.512126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c4d2f368-2e4b-4bdb-8dee-547b0e9a4ec9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.513325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c4d2f368-2e4b-4bdb-8dee-547b0e9a4ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.520190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 777/1547] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:47.521867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43127-36dd-44c8-81b5-8d586042e57d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.521867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43127-36dd-44c8-81b5-8d586042e57d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.522083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d43127-36dd-44c8-81b5-8d586042e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.549502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b6d43127-36dd-44c8-81b5-8d586042e57d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.550767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b6d43127-36dd-44c8-81b5-8d586042e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.578380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 771/1548] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.579824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ede25-9808-4301-9c12-e2778d6b7ef5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.580111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ede25-9808-4301-9c12-e2778d6b7ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.580383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ede25-9808-4301-9c12-e2778d6b7ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.599402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e81ede25-9808-4301-9c12-e2778d6b7ef5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.600610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e81ede25-9808-4301-9c12-e2778d6b7ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.619576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 778/1549] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.620739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.621024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.621298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.641079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.642316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.650747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 779/1550] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.651547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.660896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab630fea-157c-4cdb-9750-803cfed3f70a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.665930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4950a27-66a4-47f0-a88d-cf37d4b2ee81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.666713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4950a27-66a4-47f0-a88d-cf37d4b2ee81 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.666996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4950a27-66a4-47f0-a88d-cf37d4b2ee81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.667265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4950a27-66a4-47f0-a88d-cf37d4b2ee81 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.667897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4950a27-66a4-47f0-a88d-cf37d4b2ee81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.669802 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4950a27-66a4-47f0-a88d-cf37d4b2ee81 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.670374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 772/1551] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.671691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.671975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.672245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.691623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.692831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.701286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 773/1552] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.702024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.711265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160727f-c7b5-4436-894b-60d577524c14 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.716439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.717014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.717291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.717559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.718174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.719228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:32:47.719228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:32:47.719228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:32:47.720409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55ba64-351b-459b-a218-3b503d4c4925 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.751845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 780/1553] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:47.753137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c14b-db6e-4898-a26a-89849e8b9af7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.753458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c14b-db6e-4898-a26a-89849e8b9af7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.753735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c14b-db6e-4898-a26a-89849e8b9af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.773446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9d54c14b-db6e-4898-a26a-89849e8b9af7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.774692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9d54c14b-db6e-4898-a26a-89849e8b9af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.783429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 781/1554] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.784637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bba565-1ce7-4ee4-99e2-d682312cb5ac None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.784927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bba565-1ce7-4ee4-99e2-d682312cb5ac None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.785231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bba565-1ce7-4ee4-99e2-d682312cb5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.808954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d6bba565-1ce7-4ee4-99e2-d682312cb5ac None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.808954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d6bba565-1ce7-4ee4-99e2-d682312cb5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.814813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 782/1555] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.815993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d35569-3256-46d3-bebc-950a7671ef14 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.816278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d35569-3256-46d3-bebc-950a7671ef14 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.816553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d35569-3256-46d3-bebc-950a7671ef14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.833208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 774/1556] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/users => generated 291 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:47.838956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42cb20-55c0-43d7-95cf-b5f5c2778722 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.838956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42cb20-55c0-43d7-95cf-b5f5c2778722 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.838956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42cb20-55c0-43d7-95cf-b5f5c2778722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.839244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e5d35569-3256-46d3-bebc-950a7671ef14 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.839897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e5d35569-3256-46d3-bebc-950a7671ef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.847898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 783/1557] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.849278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.849561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.849834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.857205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2f42cb20-55c0-43d7-95cf-b5f5c2778722 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.866471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2f42cb20-55c0-43d7-95cf-b5f5c2778722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.867717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 775/1558] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.869102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cab44-d687-4e2d-8f9e-c420b84bfede None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.869386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cab44-d687-4e2d-8f9e-c420b84bfede None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.869659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cab44-d687-4e2d-8f9e-c420b84bfede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.877873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.879130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.887981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 784/1559] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.888867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.890853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-da8cab44-d687-4e2d-8f9e-c420b84bfede None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.892154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-da8cab44-d687-4e2d-8f9e-c420b84bfede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.898738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b79fc4-4bab-4ad2-8894-57fcdeb939d7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.902411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 776/1560] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.903500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.903791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.904070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.904386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.905444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.905782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.906415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.907143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.908772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.918708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:47.922693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 785/1561] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:47.922693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.925415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:47.926689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:47.930657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c6aa3-d687-4ca9-805b-32ead5d00bb3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.935112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 777/1562] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:47.935333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.935898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.936000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.936181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.936448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.937219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:47.937514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.939140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.949129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b78424-fffb-4ec7-b9d0-ac512cc7dbe3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.952187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 786/1563] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:47.953051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.958126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.958764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.959052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.959504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.959976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.961621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.968487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0aece-b888-40a4-a050-71dbd9840362 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.974170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 778/1564] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:47.974298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.980459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:47.980591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.980591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.980591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.980591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:47.980591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:47.980591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:47.984484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f78888-fe7e-4c3a-be85-36e9ec2a8330 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:47.989332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:47.989987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:47.990268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:47.990697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:47.991339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:47.991641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.000687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 787/1565] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:48.000863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.000965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.005065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ac278-4b73-41bf-b812-8bd7a8fdaec1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.010201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 779/1566] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:48.010201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.020697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.020697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.020697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.020697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.021080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:48.021080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.021080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.021343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b813c-183c-4189-b2ca-f5ffc0515eb0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.025150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.026070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.026373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.026650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.026778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 788/1567] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:48.027339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:48.027650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.028286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.029257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.034608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:48.039881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca1be1-388a-4846-b771-6918d756644b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.045328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 780/1568] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:48.045328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.050454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.051182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.051550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.051827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.052473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.054482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.060772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081950c-d534-40f3-8bb6-084ea5760d2f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.061333 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.061928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.062208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.062515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.063152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:48.063451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.065064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.066000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 789/1569] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:48.067095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.077971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 781/1570] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:48.078138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397717e4-8a29-47d5-8621-d8ef86c2644b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.078717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.091445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.092049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.092331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.092603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.093273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.095276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.096932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f116217-8a2e-4598-806c-5a9a728048ac admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.101773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.102409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.102695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.102996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.103945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.105557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.115518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 782/1571] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:48.120462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.126541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a82ada-38c6-45cc-b182-38a930d139a9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.131282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.131861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.132144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.132460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.133078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.135973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.227489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 790/1572] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.228978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.258664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 783/1573] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.258664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.294731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afc801-0f38-4f2e-82d6-d19db1251350 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.318558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfc481-ce22-4813-99d5-ad1bb0ef0e09 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.323721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.324330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.324618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.324886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.325518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.327463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.420141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.420752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.421035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.421188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 784/1574] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.421364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.422007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.422145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.423982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.485808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dcd48-de42-4593-a060-a7448542590e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.490493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.491099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.491452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.491783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.492414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.494374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.499607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:48.514907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 791/1575] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/projects => generated 357 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.515657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.577062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de66ee1-d668-4c37-9238-9d4efca7f0e8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.581767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.582467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.582748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.583016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] PATH_INFO: `/v3/projects/c0a44ea5393c4fb58fce35e8cbc4103b/users/d73c9c5108a845fdbd25b67bcb3a00f4/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.586025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:48.587179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 785/1576] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/projects => generated 391 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.589514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.599169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a44ea5393c4fb58fce35e8cbc4103b, user_id=d73c9c5108a845fdbd25b67bcb3a00f4, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.600251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:32:48.600251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:32:48.600251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:32:48.602245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.650495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a4f29-f1fb-4bc3-bd97-688b0036770a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.655686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.656289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.656571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.656840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.657458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.659692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.665615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 792/1577] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:48 2022] PUT /identity/v3/projects/c0a44ea5393c4fb58fce35e8cbc4103b/users/d73c9c5108a845fdbd25b67bcb3a00f4/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:48.666616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.728803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819a1d-58a2-4d8d-b4fe-7648aba43627 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.729916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 786/1578] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.730894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.787942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.788525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.788806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.789079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] PATH_INFO: `/v3/projects/61aba223b08e436db75b7b66ec486aa9/users/c74bbd46fb3743ff86ca46f33629e6ae/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.792550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8262e2-2d1a-4d71-8fbd-95483edb6153 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.797265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.797953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.798308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.798662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.799358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.801327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.808519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61aba223b08e436db75b7b66ec486aa9, user_id=c74bbd46fb3743ff86ca46f33629e6ae, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.812558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.909247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 787/1579] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:48.909247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.917473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742190f5-0fac-4ee7-bbe1-601b2c3c4ae9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.917775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 793/1580] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:48 2022] PUT /identity/v3/projects/61aba223b08e436db75b7b66ec486aa9/users/c74bbd46fb3743ff86ca46f33629e6ae/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:48.919346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:48.977454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.978878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.979341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.979605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.980229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.982200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:48.988326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3d3b-d8c8-4297-90fb-8c943c343f9a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:48.990991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:48.993867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:48.994503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:48.994783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:48.995057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:48.995683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:48.997596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.101904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 794/1581] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:49.102111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 788/1582] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:48 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:49.102111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.102307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.108800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:49.112756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebc5f4-8137-4fad-9cc8-04811ae3d846 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.113538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489762b-5c70-450d-9bbe-be2d7771a913 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.117611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.118112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e364fd1-6b38-421b-893a-355f46688021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.118497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.118756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e364fd1-6b38-421b-893a-355f46688021 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.119039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e364fd1-6b38-421b-893a-355f46688021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.119315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e364fd1-6b38-421b-893a-355f46688021 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.119421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.119810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.119936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e364fd1-6b38-421b-893a-355f46688021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.120699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.121935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e364fd1-6b38-421b-893a-355f46688021 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.122794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.216076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 789/1583] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:49 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:49.217328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.217642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.218256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.240493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 795/1584] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:49 2022] POST /identity/v3/users => generated 279 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:49.241027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:49.241147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.241247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:49.248008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 790/1585] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:49.248417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a4749-9992-43fc-8672-8303d621c205 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.248790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.253903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.254895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.255175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.255559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.256296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.258308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0cb6d-580a-43bc-a9fc-749343f07af4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.261734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.263441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7356b3-a971-4737-83d0-c2048c47f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.264040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7356b3-a971-4737-83d0-c2048c47f088 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.264316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7356b3-a971-4737-83d0-c2048c47f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.264582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7356b3-a971-4737-83d0-c2048c47f088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.265504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7356b3-a971-4737-83d0-c2048c47f088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.267136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7356b3-a971-4737-83d0-c2048c47f088 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.277878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 791/1586] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.279083 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.279367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.279639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.306731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] MFA Rules not processed for user `c74bbd46fb3743ff86ca46f33629e6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:49.307928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:49.347367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 796/1587] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:49 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:49.348661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.359163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94aabf-29ae-4388-abb3-42ae68243905 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.363068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.363657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.363938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.364206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.364840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.366471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.375990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 792/1588] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:49.376538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 797/1589] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.376834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.377378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.387541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119fa11c-c356-4485-b7d1-4011daaf8d1b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.387699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f197-052b-409e-8403-876ba762e5f0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.392022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.392593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.392737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.392884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.393164 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.393314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.393430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.393699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.393811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.394495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.395384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.396045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.405991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 793/1590] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.406816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.409910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 798/1591] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.409910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.418673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6882c9be-e7a9-4ed2-a868-d695253ca3ae admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.424184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.424950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.425325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.425646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.426383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.428899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.439934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 794/1592] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.440798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.450713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4715d0b-23b7-49c8-bcfe-3e4a043f6887 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.455281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.455858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.456138 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.456409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.457029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.458673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.467564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f59c7-e95f-4ffe-bbaf-2a61050c9d15 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.471575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.472187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.472683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.472845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 795/1593] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.473012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.473581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.483360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.484012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a963f29-c323-40f5-92ef-b39f02ce57d0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.484842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.488787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.489421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.489701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.489970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.490635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.492228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.496326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 799/1594] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:49.497659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.506550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 796/1595] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.507518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.509424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d19e6-cae2-4629-aaa7-811ae78f49fd service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.515912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.516540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.516821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.518173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.518173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.519282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeefc99-71e4-4dd9-b505-a095709a2e93 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.519508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.526871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.527498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.527908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.528422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.529456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.532003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.535880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 800/1596] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.535880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.549305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab32a2-0fcb-4dc2-ab1b-42c8c3ac107f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.554040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.554745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.555125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.555505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.556317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.557898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.568020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 801/1597] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.568806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.578247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa8295-fd8c-44b9-beb7-478afb771a7f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.582768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.583366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.583648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.583923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.584535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.586093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.597487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 802/1598] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.598269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.608176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e683d-2ad7-4efd-aaa4-a90ebc67f1ae admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.612741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.613244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.613525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.613796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.614443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.615979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.625742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 803/1599] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.626534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.636405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1eb052-841f-4f0e-b6d0-5b9ae7194600 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.641252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.641901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.642187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.642511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] PATH_INFO: `/v3/projects/e09bed4b9f9741a8ab110a2676baee7f/users/29b4f0836bbf4684bdc6177e42b88b20/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.650918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 797/1600] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:49 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:49.652129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.660696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e09bed4b9f9741a8ab110a2676baee7f, user_id=29b4f0836bbf4684bdc6177e42b88b20, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.663526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.696227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:49.715810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862aad5-bbe0-4d05-8975-9bfb5a570025 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.720567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.721172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.721450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.721719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.722384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.725747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.734328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 798/1601] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:49.735797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.745682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7454a-3180-4921-b622-723748aa8944 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.750753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 804/1602] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:49 2022] PUT /identity/v3/projects/e09bed4b9f9741a8ab110a2676baee7f/users/29b4f0836bbf4684bdc6177e42b88b20/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:49.751439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.801550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:49.804653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.805358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.805641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.805939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] PATH_INFO: `/v3/projects/02813cf0683749ee87b039b28f1aeebb/users/2b9ab1983bb84318b509878b930e9402/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.820377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07551a63-0c66-4111-9df3-2b7fbb2b91cd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:49.827276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:49.828035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:49.828408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:49.828824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] PATH_INFO: `/v3/projects/53cde5b369084772897d46c38b083d8f/users/ecf076dde874415aa734644f27a10b4b/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:49.830305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=02813cf0683749ee87b039b28f1aeebb, user_id=2b9ab1983bb84318b509878b930e9402, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.833583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.846751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53cde5b369084772897d46c38b083d8f, user_id=ecf076dde874415aa734644f27a10b4b, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:49.849567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:49.906358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 799/1603] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:49 2022] PUT /identity/v3/projects/02813cf0683749ee87b039b28f1aeebb/users/2b9ab1983bb84318b509878b930e9402/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:49.907447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.936030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 805/1604] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:49 2022] PUT /identity/v3/projects/53cde5b369084772897d46c38b083d8f/users/ecf076dde874415aa734644f27a10b4b/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:49.941026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:49.973956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a50bd1-9348-4313-899c-21a0ceca1dac admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.005765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f4d4-315e-471d-b0b7-da93fd292064 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.012435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.013306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.013661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.014008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] PATH_INFO: `/v3/projects/2f837df8976a4f8dbccd2b34b341acf5/users/455815f31a444487af7bdd7ef8ae81fc/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.033078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.033747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.034028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.034301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] PATH_INFO: `/v3/projects/d93c5a3ef20c4a5b87af4803f820b60e/users/409c324dca974d9dad9529b2a0d82b30/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.038210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f837df8976a4f8dbccd2b34b341acf5, user_id=455815f31a444487af7bdd7ef8ae81fc, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.042021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.046559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:50.050714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93c5a3ef20c4a5b87af4803f820b60e, user_id=409c324dca974d9dad9529b2a0d82b30, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.053460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.138463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 800/1606] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:49 2022] PUT /identity/v3/projects/d93c5a3ef20c4a5b87af4803f820b60e/users/409c324dca974d9dad9529b2a0d82b30/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:50.138648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 806/1606] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:49 2022] PUT /identity/v3/projects/2f837df8976a4f8dbccd2b34b341acf5/users/455815f31a444487af7bdd7ef8ae81fc/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:50.138766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.138856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.215723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae32f3-9b47-4319-a124-2b01aa48cd1f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.220887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333d04d-868c-4cf8-b7f3-a69d76b2505d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.221500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333d04d-868c-4cf8-b7f3-a69d76b2505d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.221869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333d04d-868c-4cf8-b7f3-a69d76b2505d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.222051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333d04d-868c-4cf8-b7f3-a69d76b2505d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.222725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333d04d-868c-4cf8-b7f3-a69d76b2505d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.224706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333d04d-868c-4cf8-b7f3-a69d76b2505d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.237501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4b50-b7c6-482b-86d2-b53934df8d35 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.242656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4fa444-bd4e-4176-ac5f-1c5e4fac42bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.243326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fa444-bd4e-4176-ac5f-1c5e4fac42bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.244129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fa444-bd4e-4176-ac5f-1c5e4fac42bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.244400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4fa444-bd4e-4176-ac5f-1c5e4fac42bf admin admin] PATH_INFO: `/v3/projects/46cc32edc42d42399a330991d62f4682/users/c5bef358e02645d98ef923c8e131d80b/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.260367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fa444-bd4e-4176-ac5f-1c5e4fac42bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=46cc32edc42d42399a330991d62f4682, user_id=c5bef358e02645d98ef923c8e131d80b, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.263092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4fa444-bd4e-4176-ac5f-1c5e4fac42bf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.340597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 801/1607] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/users => generated 296 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:50.342302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b4683-2465-4187-899c-a6229ef889f2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.342714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b4683-2465-4187-899c-a6229ef889f2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.343089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b4683-2465-4187-899c-a6229ef889f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.362726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-747b4683-2465-4187-899c-a6229ef889f2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.364006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-747b4683-2465-4187-899c-a6229ef889f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.372672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 802/1608] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:50.374055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9a4fc-0553-428e-9a4e-0b105f6ec65f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.374366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9a4fc-0553-428e-9a4e-0b105f6ec65f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.374639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9a4fc-0553-428e-9a4e-0b105f6ec65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.410437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8cb9a4fc-0553-428e-9a4e-0b105f6ec65f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.410437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8cb9a4fc-0553-428e-9a4e-0b105f6ec65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.410758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 807/1609] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:50 2022] PUT /identity/v3/projects/46cc32edc42d42399a330991d62f4682/users/c5bef358e02645d98ef923c8e131d80b/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:50.410758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07553bde-0a91-4ae5-837a-241af2295a3c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.410758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07553bde-0a91-4ae5-837a-241af2295a3c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.410758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07553bde-0a91-4ae5-837a-241af2295a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.420359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-07553bde-0a91-4ae5-837a-241af2295a3c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.421599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-07553bde-0a91-4ae5-837a-241af2295a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.456041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:50.482608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 803/1610] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:50.482608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.482608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.482608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.503255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 808/1611] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:50.509343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120726e9-7ea0-49e8-8acc-17b17b8a7765 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.509343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120726e9-7ea0-49e8-8acc-17b17b8a7765 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.509343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120726e9-7ea0-49e8-8acc-17b17b8a7765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.519167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.520128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.530504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 804/1612] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:50.531216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.541060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e10e-8202-4bdd-8e06-414a17391111 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.543827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-120726e9-7ea0-49e8-8acc-17b17b8a7765 None None] MFA Rules not processed for user `455815f31a444487af7bdd7ef8ae81fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.545073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-120726e9-7ea0-49e8-8acc-17b17b8a7765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.546048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.546689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.546965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.547229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.547948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.549551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.560620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 805/1613] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:50.561342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.623621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 809/1614] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:50.623621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.623621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.623621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.629075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400425-92a0-404c-bc7f-cc4641ae250f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.635408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.636310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.636735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.637020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.647501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.648959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.656870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] MFA Rules not processed for user `c5bef358e02645d98ef923c8e131d80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.657024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:50.657124 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.663327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 806/1615] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:50.664182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.674918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63c25d-1e6b-4935-a5bc-8aa51b3c0740 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.679336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.680592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.681015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.681407 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.694071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.697166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.707076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 807/1616] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:50.707899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.716222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb840e2-d6bf-4b40-916b-e296f5ec58b3 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.721515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.725750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.725750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.725750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.733961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.735467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.736042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 810/1617] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:50.741295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.747818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 808/1618] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:50.748962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.758063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12950bb3-e0a5-4f77-9809-758005c3c8d6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.758669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7990ef-26ef-43c4-b651-79c0660fb0cf service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.763068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.763698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.764079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.764354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.764550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c75c6-ab12-4b03-a916-19b3147b04e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.765453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c75c6-ab12-4b03-a916-19b3147b04e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.765739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c75c6-ab12-4b03-a916-19b3147b04e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.766014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c75c6-ab12-4b03-a916-19b3147b04e6 admin admin] PATH_INFO: `/v3/projects/836485dfa6114c6a943dfe25f3c84d9a/users/33359f79c6384463befe2b11a01ec4d8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.775865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.777382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.784599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c75c6-ab12-4b03-a916-19b3147b04e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836485dfa6114c6a943dfe25f3c84d9a, user_id=33359f79c6384463befe2b11a01ec4d8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.787727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c75c6-ab12-4b03-a916-19b3147b04e6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.789198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 811/1619] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:50.790087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.801132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019e1f4-644c-466d-8d3b-2b971956bfce service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:50.806313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c441dd-7369-49d3-9221-edc9e022942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:50.807011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c441dd-7369-49d3-9221-edc9e022942b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.807457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c441dd-7369-49d3-9221-edc9e022942b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.808249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c441dd-7369-49d3-9221-edc9e022942b admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.808565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c441dd-7369-49d3-9221-edc9e022942b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:50.810847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c441dd-7369-49d3-9221-edc9e022942b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:50.869625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 809/1620] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:50 2022] PUT /identity/v3/projects/836485dfa6114c6a943dfe25f3c84d9a/users/33359f79c6384463befe2b11a01ec4d8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:50.871378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502d9f-2318-4ccd-89d2-fa26871bfeaf None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.871697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502d9f-2318-4ccd-89d2-fa26871bfeaf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.872684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2502d9f-2318-4ccd-89d2-fa26871bfeaf None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.873994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 810/1621] 10.222.0.13 () {54 vars in 962 bytes} [Sat Sep 3 01:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:32:50.874769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502d9f-2318-4ccd-89d2-fa26871bfeaf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:50.920106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 812/1622] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/projects => generated 355 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:50.921524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:50.921640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:50.922011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:50.959805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:50.973325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:50.976471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:50.995906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2502d9f-2318-4ccd-89d2-fa26871bfeaf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.068396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.069204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.069326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.069572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.070473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.073186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.087009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 811/1623] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:51.088009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.102683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f11e8-0bbc-48c6-b875-bddef86cb998 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.108094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae9947-0561-4aa2-8e34-dcfd67c99ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.108720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9947-0561-4aa2-8e34-dcfd67c99ad8 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.109003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9947-0561-4aa2-8e34-dcfd67c99ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.109284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9947-0561-4aa2-8e34-dcfd67c99ad8 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.109994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae9947-0561-4aa2-8e34-dcfd67c99ad8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.112220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae9947-0561-4aa2-8e34-dcfd67c99ad8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.126012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 813/1624] 10.222.0.13 () {58 vars in 1059 bytes} [Sat Sep 3 01:32:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:51.127005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.139751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b696f5-86a8-4783-a169-124ded35c4ae None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.145290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.152766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.152766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.152766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] PATH_INFO: `/v3/projects/836485dfa6114c6a943dfe25f3c84d9a/users/33359f79c6384463befe2b11a01ec4d8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.152766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836485dfa6114c6a943dfe25f3c84d9a, user_id=33359f79c6384463befe2b11a01ec4d8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.152766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.173038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 814/1625] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:51 2022] PUT /identity/v3/projects/836485dfa6114c6a943dfe25f3c84d9a/users/33359f79c6384463befe2b11a01ec4d8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:51.174050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.230487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 812/1626] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:32:51 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:51.232741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.233145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.233955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.283818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] MFA Rules not processed for user `33359f79c6384463befe2b11a01ec4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:51.285483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:51.297276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f7ba8-e05b-4098-b013-9af3fdc23933 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.303854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.304534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.304820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.305144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.314150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:51.389345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.391331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.406662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 815/1627] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:51.407751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.433507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 813/1628] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:51.449973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.507371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0277e-7375-45d7-a141-b33e3460ff42 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.519857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.522013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.523546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.523816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.525302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.532121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.566995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 816/1629] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:51.570661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.584313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffd7a7-a663-4013-92af-09640cd63a66 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.592581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.593841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.594192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.594562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.596618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a8878-d9eb-495d-9d9d-5bfb4a247dbb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.607069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20bab0-dd3c-4ce0-8cbf-38a2117254c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.608276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20bab0-dd3c-4ce0-8cbf-38a2117254c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.609787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.609902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20bab0-dd3c-4ce0-8cbf-38a2117254c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.610869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.611075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20bab0-dd3c-4ce0-8cbf-38a2117254c0 admin admin] PATH_INFO: `/v3/projects/8dca69797b90429aba416d0e51692a15/users/955b0777314044df86e118ec12238620/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.635479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 814/1630] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:32:51.638251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20bab0-dd3c-4ce0-8cbf-38a2117254c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dca69797b90429aba416d0e51692a15, user_id=955b0777314044df86e118ec12238620, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.640458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20bab0-dd3c-4ce0-8cbf-38a2117254c0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:51.733000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 817/1631] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:51 2022] PUT /identity/v3/projects/8dca69797b90429aba416d0e51692a15/users/955b0777314044df86e118ec12238620/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:51.736715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:51.751448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326ff95-7675-47b0-9bd7-e94f79523b27 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:51.830049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:51.886550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbbf3c-6bef-4014-8289-9f04d13e04b9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.887444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbbf3c-6bef-4014-8289-9f04d13e04b9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.887744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbbf3c-6bef-4014-8289-9f04d13e04b9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.889287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 818/1632] 10.222.0.13 () {54 vars in 912 bytes} [Sat Sep 3 01:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:32:51.898563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.898869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.899179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.944338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:51.945773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:51.984436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:51.986534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:51.987377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:51.988606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:51.989923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:51.998642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:52.025356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 815/1633] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:32:52.027629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:52.050476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9878-b765-42f3-a3ef-b5c8bc9b1069 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:52.061995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e911a-d71b-433b-b220-2c6e834a2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:52.065304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e911a-d71b-433b-b220-2c6e834a2cce admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:52.065817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e911a-d71b-433b-b220-2c6e834a2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:52.066426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e911a-d71b-433b-b220-2c6e834a2cce admin admin] PATH_INFO: `/v3/projects/8dca69797b90429aba416d0e51692a15/users/955b0777314044df86e118ec12238620/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:52.072299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e911a-d71b-433b-b220-2c6e834a2cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dca69797b90429aba416d0e51692a15, user_id=955b0777314044df86e118ec12238620, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:52.082063 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e911a-d71b-433b-b220-2c6e834a2cce admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:52.115924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 816/1634] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:32:52 2022] PUT /identity/v3/projects/8dca69797b90429aba416d0e51692a15/users/955b0777314044df86e118ec12238620/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:52.119682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 819/1635] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:32:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:52.136401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:52.136998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:52.137575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:52.138794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:52.178583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] MFA Rules not processed for user `955b0777314044df86e118ec12238620`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:52.180479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:52.261749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:52.275765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f23464-79e4-4c12-abd7-267cb7ea0941 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:52.347301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:52.348563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:52.348958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:52.349445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:52.374386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 817/1636] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:52.441809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:52.443709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:52.458964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 820/1637] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:52.499555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:53.307179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:53.307179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb2388-0df5-4f96-9570-2df92a2d430e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:53.307179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:53.307179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:53.310274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:53.310274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:53.310274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:53.310274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:53.310274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 818/1638] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:53.392255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:53.581743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e68f8b-1ad7-42eb-abed-e1360b3e4946 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:53.586720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:53.589055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33359f79c6384463befe2b11a01ec4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '836485dfa6114c6a943dfe25f3c84d9a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:53.589776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:53.590188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:53.591168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:53.592035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:53.592468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:53.594604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:53.598567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4788d49-88eb-41ee-9424-fa707eabd5e1 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:53.601035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:53.601813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:53.602262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:53.602714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:53.614852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 821/1639] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:53.616191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:53.652170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39c9d3-fe37-4cb4-bd1e-6a83edc2e7c6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:53.669546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33359f79c6384463befe2b11a01ec4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '836485dfa6114c6a943dfe25f3c84d9a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:53.670294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:53.670646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:53.670930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:53.671653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:53.677147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:53.733591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:53.737894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:53.808146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 819/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:53.860226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 822/1641] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:32:53 2022] POST /identity/v3/projects => generated 355 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:32:53.866582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:53.891077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2cc4f-3b93-4c43-828f-6ff73f1868a7 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:54.121468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33359f79c6384463befe2b11a01ec4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '836485dfa6114c6a943dfe25f3c84d9a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:54.123296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.123296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.123296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.123568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:32:54.123839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:54.128090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fc427-2bcb-4719-a8d6-43881de502be tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:54.179722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 820/1642] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:32:54.261384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:54.287579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ac5d6-0361-44f5-9643-16fa3118e75c tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:54.303428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33359f79c6384463befe2b11a01ec4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '836485dfa6114c6a943dfe25f3c84d9a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:54.306666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.306964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.307257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] PATH_INFO: `/v3/projects/7e7883d4887a42cba2580952d855d639` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.311282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7e7883d4887a42cba2580952d855d639)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:54.315970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:54.325662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8106-1d29-493a-b2a6-e36ddf9e19f0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.326228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8106-1d29-493a-b2a6-e36ddf9e19f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.326314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8106-1d29-493a-b2a6-e36ddf9e19f0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.328378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 821/1643] 10.222.0.13 () {54 vars in 962 bytes} [Sat Sep 3 01:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:32:54.335671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099056b-05de-4387-bb2e-7b0771f92357 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.335969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099056b-05de-4387-bb2e-7b0771f92357 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.336246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099056b-05de-4387-bb2e-7b0771f92357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.384609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2099056b-05de-4387-bb2e-7b0771f92357 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:54.386187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2099056b-05de-4387-bb2e-7b0771f92357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:54.498612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:54.504941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:54.512448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7883d4887a42cba2580952d855d639', 'request_id': 'req-29d14d3f-6da5-4f16-b72c-4c72d91737d6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:54.549750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 822/1644] 10.222.0.13 () {58 vars in 1059 bytes} [Sat Sep 3 01:32:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:54.557003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb9970-17f1-457c-91e0-bba0f50c326a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.557466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb9970-17f1-457c-91e0-bba0f50c326a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.557841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb9970-17f1-457c-91e0-bba0f50c326a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.559752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 823/1645] 10.222.0.13 () {54 vars in 912 bytes} [Sat Sep 3 01:32:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:32:54.565362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.565813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.566186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.596841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:54.598321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:54.608863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7883d4887a42cba2580952d855d639', 'request_id': 'req-29d14d3f-6da5-4f16-b72c-4c72d91737d6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:54.616511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7883d4887a42cba2580952d855d639', 'request_id': 'req-29d14d3f-6da5-4f16-b72c-4c72d91737d6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:54.641667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] The token cache is being invalidate because project 7e7883d4887a42cba2580952d855d639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:32:54.641967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:54.643492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:54.647070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 823/1646] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:32:54 2022] DELETE /identity/v3/projects/7e7883d4887a42cba2580952d855d639 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:54.760401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 824/1647] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:32:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:32:54.765445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:54.767060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:54.865025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d14d3f-6da5-4f16-b72c-4c72d91737d6 tempest-VolumeQuotaClassesTest-541353875 tempest-VolumeQuotaClassesTest-541353875-project-admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:54.872698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219dee7-cb14-4175-b8e8-b70f1ec1d47e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:54.872987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219dee7-cb14-4175-b8e8-b70f1ec1d47e service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:54.873871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219dee7-cb14-4175-b8e8-b70f1ec1d47e service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:54.874161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219dee7-cb14-4175-b8e8-b70f1ec1d47e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:54.892801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219dee7-cb14-4175-b8e8-b70f1ec1d47e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:54.895004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219dee7-cb14-4175-b8e8-b70f1ec1d47e service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:54.913476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 824/1648] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:32:55.520453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:55.523315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:55.661722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9dac9-a709-4529-a017-a3064d790b04 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:55.671201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:55.672122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:55.672697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:55.672798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] PATH_INFO: `/v3/users/c74bbd46fb3743ff86ca46f33629e6ae` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:55.674120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74bbd46fb3743ff86ca46f33629e6ae)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:55.678474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:56.067799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74bbd46fb3743ff86ca46f33629e6ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:56.168666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74bbd46fb3743ff86ca46f33629e6ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:56.169497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74bbd46fb3743ff86ca46f33629e6ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:56.193005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74bbd46fb3743ff86ca46f33629e6ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:56.195607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:56.207199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74bbd46fb3743ff86ca46f33629e6ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:56.222358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 825/1649] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:32:55 2022] DELETE /identity/v3/users/c74bbd46fb3743ff86ca46f33629e6ae => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:56.226584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:56.227099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:56.227305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:56.263416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:32:56.265630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:56.457597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 825/1650] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:32:56.530463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:56.533425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:56.799385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27028a-2290-4970-9d24-36b7e447f401 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:56.803988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:56.808634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:56.808634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:56.808634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:56.841111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:56.843118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:56.869803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 826/1651] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:32:58.520917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:58.523979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:32:58.543554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc63f3-2da9-440f-9446-1406c53c8998 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:58.551097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:58.553483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:58.553944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:58.554459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] PATH_INFO: `/v3/projects/61aba223b08e436db75b7b66ec486aa9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:58.556121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61aba223b08e436db75b7b66ec486aa9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:58.560646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:58.831005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61aba223b08e436db75b7b66ec486aa9', 'request_id': 'req-8d449f84-0054-4109-8279-75d5b7f88523'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:58.929235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61aba223b08e436db75b7b66ec486aa9', 'request_id': 'req-8d449f84-0054-4109-8279-75d5b7f88523'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:58.937102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61aba223b08e436db75b7b66ec486aa9', 'request_id': 'req-8d449f84-0054-4109-8279-75d5b7f88523'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:58.955950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:32:58.961074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] The token cache is being invalidate because project 61aba223b08e436db75b7b66ec486aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:32:58.961549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:58.965147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:58.971259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 826/1652] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:32:58 2022] DELETE /identity/v3/projects/61aba223b08e436db75b7b66ec486aa9 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:32:58.976185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:32:58.989399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8948db-f615-49fe-b9f5-4fc3735fa7d6 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:32:59.126979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:32:59.127796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:32:59.127954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:32:59.128231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] PATH_INFO: `/v3/users/33359f79c6384463befe2b11a01ec4d8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:32:59.129427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33359f79c6384463befe2b11a01ec4d8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:32:59.131928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:32:59.535603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33359f79c6384463befe2b11a01ec4d8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:59.637412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33359f79c6384463befe2b11a01ec4d8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:59.638261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33359f79c6384463befe2b11a01ec4d8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:59.668854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33359f79c6384463befe2b11a01ec4d8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:59.684516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ddac6614-5fb6-4806-a78e-318a4c6eb578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33359f79c6384463befe2b11a01ec4d8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:32:59.702839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 827/1653] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:32:58 2022] DELETE /identity/v3/users/33359f79c6384463befe2b11a01ec4d8 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:02.092996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:02.098442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:02.122995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d449f84-0054-4109-8279-75d5b7f88523 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:02.244863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:02.245771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:02.246192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:02.246734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] PATH_INFO: `/v3/projects/836485dfa6114c6a943dfe25f3c84d9a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:02.248293 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=836485dfa6114c6a943dfe25f3c84d9a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:02.250992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:02.443411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836485dfa6114c6a943dfe25f3c84d9a', 'request_id': 'req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:02.545207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836485dfa6114c6a943dfe25f3c84d9a', 'request_id': 'req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:02.553567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836485dfa6114c6a943dfe25f3c84d9a', 'request_id': 'req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:02.594780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] The token cache is being invalidate because project 836485dfa6114c6a943dfe25f3c84d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:02.609750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:02.611273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-27b7e268-e461-4fd4-a5ee-3c29e1fc95c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:02.623915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 827/1654] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:02 2022] DELETE /identity/v3/projects/836485dfa6114c6a943dfe25f3c84d9a => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:02.636366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:02.636366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:02.636366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:02.671531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:02.673053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:02.701464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:02.800238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 828/1655] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:02.809629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:02.810142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:02.810848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:02.857943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:02.859590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:02.873770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 828/1656] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:02.881641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:02.902062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db348a7-5177-41cd-8afc-03f5f2d5fc3a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:02.910635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:02.911695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:02.913632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:02.914206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:02.915156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:33:02.916326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:02.920322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:02.942819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 829/1657] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:33:02.950417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:02.966262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ff15a-8948-4580-8f7b-e702956fb302 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:02.985625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:02.986938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:02.987231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:02.988197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:02.989347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:02.995971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:03.136549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 829/1658] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:02 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:03.141478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:03.158597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab09fd-191e-4599-a3e2-f326f64cb1a2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:03.410533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:03.411106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:03.411393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:03.411674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:03.412798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:03.418727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:03.574606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 830/1659] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:03 2022] POST /identity/v3/users => generated 291 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:03.575591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:03.589631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548857d6-35b8-4677-930a-1be7b9ea9f9e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:03.604177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:03.605172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:03.605617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:03.608314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:03.608314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:03.609751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:03.649427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 830/1660] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:03.653633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:03.668513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa563c05-61f7-4b37-9972-f407bf61434f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:03.683047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04190f-565d-4f9b-8383-0a37c58e19ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:03.683686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04190f-565d-4f9b-8383-0a37c58e19ef admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:03.683973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04190f-565d-4f9b-8383-0a37c58e19ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:03.687253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04190f-565d-4f9b-8383-0a37c58e19ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:03.687997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04190f-565d-4f9b-8383-0a37c58e19ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:03.689955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04190f-565d-4f9b-8383-0a37c58e19ef admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:03.745001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:03.747091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 831/1661] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:03.759709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de333fa-520a-46e7-8f96-dedf58b58c15 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:03.772849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b0907-fc95-4a43-bb88-e491797cadad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:03.773613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b0907-fc95-4a43-bb88-e491797cadad admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:03.773908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b0907-fc95-4a43-bb88-e491797cadad admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:03.781429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b0907-fc95-4a43-bb88-e491797cadad admin admin] PATH_INFO: `/v3/projects/c39d18fcaa4740529a933f94a25244b3/users/97f58bc2304d40f18f4eb4d9c22c55ff/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:03.835059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b0907-fc95-4a43-bb88-e491797cadad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c39d18fcaa4740529a933f94a25244b3, user_id=97f58bc2304d40f18f4eb4d9c22c55ff, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:03.840934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b0907-fc95-4a43-bb88-e491797cadad admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:04.011203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 831/1662] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:03 2022] PUT /identity/v3/projects/c39d18fcaa4740529a933f94a25244b3/users/97f58bc2304d40f18f4eb4d9c22c55ff/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:04.025856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:04.027148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:04.027682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:04.110486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] MFA Rules not processed for user `97f58bc2304d40f18f4eb4d9c22c55ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:04.112112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:04.349704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811ddcc-0ba5-4c4c-b1ba-9888fb368d9d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:04.350747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811ddcc-0ba5-4c4c-b1ba-9888fb368d9d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:04.351121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811ddcc-0ba5-4c4c-b1ba-9888fb368d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:04.364264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 832/1663] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:04.373402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:04.439604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5811ddcc-0ba5-4c4c-b1ba-9888fb368d9d None None] MFA Rules not processed for user `d73c9c5108a845fdbd25b67bcb3a00f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:04.444088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5811ddcc-0ba5-4c4c-b1ba-9888fb368d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:04.496882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:04.500792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1580eb6-acd8-4d43-8767-d639196db1a1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:04.511036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:04.512012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:04.512495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:04.514565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:04.515454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:04.521688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:04.652760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 832/1664] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:04.663955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:04.664532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:04.665372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:04.705971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] MFA Rules not processed for user `d73c9c5108a845fdbd25b67bcb3a00f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:04.707320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:04.721599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 833/1665] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:04 2022] POST /identity/v3/projects => generated 381 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:04.723477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:04.729476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 833/1666] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:04.739840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:04.741282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:04.930093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:04.941874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10aae2-ede6-4f87-bb72-724cff588d14 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:04.969490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:04.970149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:04.970552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:04.970952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:04.972099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:04.983043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:04.988841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5f8bf-1e81-4d69-b3de-e46f6a6c567e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:05.132315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 834/1667] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:04 2022] POST /identity/v3/users => generated 297 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:05.135386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:05.145115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:05.186978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512534d4-20c5-4b52-b15a-2231915aef5d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:05.192504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:05.193380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:05.193806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:05.194226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:05.200706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:05.203841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:05.204282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:05.205739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] PATH_INFO: `/v3/users/c5bef358e02645d98ef923c8e131d80b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:05.214242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5bef358e02645d98ef923c8e131d80b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:05.214242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:05.368796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:05.371973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:05.407115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 834/1668] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:33:05.408204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:05.462152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233e23a-fe1f-4e9b-b946-2cd12e4bc461 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:05.484238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:05.485209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:05.489917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:05.490196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:05.493520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:05.498281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5bef358e02645d98ef923c8e131d80b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:05.502245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:05.559514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 835/1669] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:05.560496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:05.590675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1a443-a5d8-45f7-a80e-24bdb03f118f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:05.600482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:05.601425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:05.601847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:05.603068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] PATH_INFO: `/v3/projects/7b8e9de1bb5c48c29ac9a4f20c8e0afd/users/54121084facd4ea488f90037cb8dc1f0/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:05.625617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5bef358e02645d98ef923c8e131d80b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:05.626771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5bef358e02645d98ef923c8e131d80b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:05.659681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5bef358e02645d98ef923c8e131d80b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:05.662974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8e9de1bb5c48c29ac9a4f20c8e0afd, user_id=54121084facd4ea488f90037cb8dc1f0, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:05.687826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:05.700268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d34d097-5551-43a2-a46d-91aa02d66774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5bef358e02645d98ef923c8e131d80b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:05.730456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 835/1670] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:33:05 2022] DELETE /identity/v3/users/c5bef358e02645d98ef923c8e131d80b => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:05.740455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:05.740455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:05.740455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:05.775366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 836/1671] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:05 2022] PUT /identity/v3/projects/7b8e9de1bb5c48c29ac9a4f20c8e0afd/users/54121084facd4ea488f90037cb8dc1f0/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:05.779300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:05.784371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:05.801472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:05.802992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:05.975818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146e307-2e23-4dee-9ce4-ec3560c760bf admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:05.981191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9897e-e505-40df-968e-8e95cf691a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:05.982154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9897e-e505-40df-968e-8e95cf691a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:05.982704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9897e-e505-40df-968e-8e95cf691a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:05.983173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9897e-e505-40df-968e-8e95cf691a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:05.984083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9897e-e505-40df-968e-8e95cf691a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:05.986244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b9897e-e505-40df-968e-8e95cf691a04 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:05.993876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 836/1672] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:06.009111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 837/1673] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:06.012582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:06.025618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f72b3-60dd-42a5-a560-b1130c8bff4b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:06.037679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:06.040246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:06.040534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:06.040815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] PATH_INFO: `/v3/projects/7b8e9de1bb5c48c29ac9a4f20c8e0afd/users/54121084facd4ea488f90037cb8dc1f0/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:06.045754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b8e9de1bb5c48c29ac9a4f20c8e0afd, user_id=54121084facd4ea488f90037cb8dc1f0, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:06.053492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:06.110786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 837/1674] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:06 2022] PUT /identity/v3/projects/7b8e9de1bb5c48c29ac9a4f20c8e0afd/users/54121084facd4ea488f90037cb8dc1f0/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:06.124742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:06.125362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:06.125842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:06.175994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] MFA Rules not processed for user `54121084facd4ea488f90037cb8dc1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:06.177633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:06.224279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:06.283697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:06.298894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:06.300658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:06.315333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 838/1675] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:06.328385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:06.334464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:06.471565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d6f37-a989-4b13-8947-238d12fd20cf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:06.474927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ddd66-05c5-4470-821e-77031b7dec7c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:06.482178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755c272-d757-4965-8951-286c3ab5ac56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:06.483396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755c272-d757-4965-8951-286c3ab5ac56 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:06.483396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755c272-d757-4965-8951-286c3ab5ac56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:06.485248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755c272-d757-4965-8951-286c3ab5ac56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:06.489548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:06.493609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:06.494100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:06.494558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:06.529763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:06.533923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:06.572383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 839/1676] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:33:06.638107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755c272-d757-4965-8951-286c3ab5ac56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:06.640444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755c272-d757-4965-8951-286c3ab5ac56 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:06.660458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 838/1677] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:33:07.121293 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:07.668499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:07.688229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b21e9-e80a-4c6b-93d0-165051a30787 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:07.704969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:07.704969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:07.704969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:07.704969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] PATH_INFO: `/v3/users/97f58bc2304d40f18f4eb4d9c22c55ff` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:07.706311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f58bc2304d40f18f4eb4d9c22c55ff)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:07.707659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:33:07.707659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 03 01:33:07.707659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: Sep 03 01:33:07.710724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:07.846071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cab6e-a529-4e53-a419-3e677207d464 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:07.846965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cab6e-a529-4e53-a419-3e677207d464 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:07.847423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7cab6e-a529-4e53-a419-3e677207d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:07.936266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8e7cab6e-a529-4e53-a419-3e677207d464 None None] MFA Rules not processed for user `29b4f0836bbf4684bdc6177e42b88b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:07.937506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8e7cab6e-a529-4e53-a419-3e677207d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.064679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f58bc2304d40f18f4eb4d9c22c55ff'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:08.065637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f58bc2304d40f18f4eb4d9c22c55ff'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:08.131831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 839/1678] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:08.133229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a61a-0ef4-44c8-b370-0b2409b7a983 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:08.133730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a61a-0ef4-44c8-b370-0b2409b7a983 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:08.134367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0a61a-0ef4-44c8-b370-0b2409b7a983 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:08.135420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 840/1679] 10.222.0.13 () {54 vars in 912 bytes} [Sat Sep 3 01:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:33:08.139233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f58bc2304d40f18f4eb4d9c22c55ff'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:08.147508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:08.147789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:08.148219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:08.171329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f58bc2304d40f18f4eb4d9c22c55ff'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:08.216066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] MFA Rules not processed for user `2b9ab1983bb84318b509878b930e9402`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:08.217304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.277269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f849e768-a087-43ea-b085-40d06a3ef58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f58bc2304d40f18f4eb4d9c22c55ff'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:08.304884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 840/1680] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:33:07 2022] DELETE /identity/v3/users/97f58bc2304d40f18f4eb4d9c22c55ff => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:08.309746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff58e-1555-4204-a07d-7d90f2cd16ec None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:08.312874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff58e-1555-4204-a07d-7d90f2cd16ec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:08.313692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff58e-1555-4204-a07d-7d90f2cd16ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:08.391391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 841/1681] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:08.392634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:08.396671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.418759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b16ff58e-1555-4204-a07d-7d90f2cd16ec None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:08.420453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b16ff58e-1555-4204-a07d-7d90f2cd16ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.472873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:08.510801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 841/1682] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:33:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:08.519611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:08.520093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:08.520380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:08.551842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:08.553666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.555943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a2827-26bd-44fd-b210-0de23d614a74 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:08.710523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 842/1683] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:08.712446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:08.714136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.729923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f95c4-eb28-45be-8d09-10e98ae84c41 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:08.738161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:08.740286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:08.741297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:08.743824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] PATH_INFO: `/v3/projects/46cc32edc42d42399a330991d62f4682` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:08.746553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46cc32edc42d42399a330991d62f4682)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:08.753153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:08.759562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:08.761591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:08.762012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:08.762366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:08.765109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:08.771242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.916782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46cc32edc42d42399a330991d62f4682', 'request_id': 'req-66d17254-42c4-49ea-9ae4-5f231bddebb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:08.925328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:08.928714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:08.943840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 842/1684] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:08.944815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:08.946846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:08.975716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67c4e1-01b3-411f-bf06-74beaad780ab service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:09.014360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46cc32edc42d42399a330991d62f4682', 'request_id': 'req-66d17254-42c4-49ea-9ae4-5f231bddebb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:09.023742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46cc32edc42d42399a330991d62f4682', 'request_id': 'req-66d17254-42c4-49ea-9ae4-5f231bddebb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:09.063618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] The token cache is being invalidate because project 46cc32edc42d42399a330991d62f4682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:09.070398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:09.078944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:09.080882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 843/1685] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:08 2022] DELETE /identity/v3/projects/46cc32edc42d42399a330991d62f4682 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:09.085148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:09.086678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.209932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:09.210597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.210877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.211244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.222159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.238797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:09.241065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74df220-73cd-4b3a-818b-ce8e37b690cf service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:09.253842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 843/1686] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:09.260513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.260513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.260513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.308896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d17254-42c4-49ea-9ae4-5f231bddebb2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:09.310036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:09.311592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.319107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:09.320765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.323994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.323994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.329724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.463056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 844/1687] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:09.470740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:09.475055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.483303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:09.485251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205536e-19f8-4d3e-a417-dbeb9d225ca5 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:09.504635 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 844/1688] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:09.506944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.506944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.507161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.541045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:09.544691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.564577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 845/1689] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:09.568575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:09.587723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05a5fd-ccaf-4458-9488-ab3f4f4f9131 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:09.598197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:09.598846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.599141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.599389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.600310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:33:09.600310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:09.602026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:09.637073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 846/1690] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:33:09.639375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:09.646802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831cd80-d4ce-4a0f-bef6-cfd5ca256927 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:09.652562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68c168-64f4-4848-be75-ddf20583f74b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:09.653429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:09.654038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.654638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.654917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.657070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.659049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7479122-3a40-4674-9623-c120e94f1761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:09.660111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7479122-3a40-4674-9623-c120e94f1761 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.660574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7479122-3a40-4674-9623-c120e94f1761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.661062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7479122-3a40-4674-9623-c120e94f1761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.661940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7479122-3a40-4674-9623-c120e94f1761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:09.664245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7479122-3a40-4674-9623-c120e94f1761 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:09.774515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:09.776508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:09.787368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 845/1691] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:09.818455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 847/1692] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:09 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:09.824903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:09.848541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f27e1-cd71-4429-9d2d-81eaf770d9ec None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.849023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f27e1-cd71-4429-9d2d-81eaf770d9ec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.849202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90025345-5af3-4c3c-bf4e-5ca80b694839 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:09.849654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f27e1-cd71-4429-9d2d-81eaf770d9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.891725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e30f27e1-cd71-4429-9d2d-81eaf770d9ec None None] MFA Rules not processed for user `409c324dca974d9dad9529b2a0d82b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:09.893942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e30f27e1-cd71-4429-9d2d-81eaf770d9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:09.989981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49edda-6ad4-4bf0-8252-ddf077f25011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:09.990644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49edda-6ad4-4bf0-8252-ddf077f25011 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:09.990929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49edda-6ad4-4bf0-8252-ddf077f25011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:09.991199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49edda-6ad4-4bf0-8252-ddf077f25011 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:09.991844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49edda-6ad4-4bf0-8252-ddf077f25011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:09.996284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49edda-6ad4-4bf0-8252-ddf077f25011 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.032488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:10.067070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 848/1693] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:10.068983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.069693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.070173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.112682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 846/1694] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:09 2022] POST /identity/v3/users => generated 287 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:10.114385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.114894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.115274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.133940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] MFA Rules not processed for user `ecf076dde874415aa734644f27a10b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:10.133940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:10.152177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] MFA Rules not processed for user `409c324dca974d9dad9529b2a0d82b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:10.153345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:10.175171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 847/1695] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:10.177927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.228087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26745e36-e7d4-4ab3-89db-ecff480178ed None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.251905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.252764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.253051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.253329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.254022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.263206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.290232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 848/1696] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:10.291458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.296488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 849/1697] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:10.307592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.319892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8933a-59b8-45dd-be55-0bae013d338b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.325235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.325964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.326253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.327283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.327960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.329643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.358091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 850/1698] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:10.368678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.383816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08df9d-7911-407e-b51b-021312727c02 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.407012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.407012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.407012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.407012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] PATH_INFO: `/v3/projects/56c93f64004b42f6889cf348b9665306/users/01e0c9e04bb84f43bf8df91cdc2f592a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.439613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea919f4-3da9-4869-8ea3-964c4b666afc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.449930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.450944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.451459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.452182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.460552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c93f64004b42f6889cf348b9665306, user_id=01e0c9e04bb84f43bf8df91cdc2f592a, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.463566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.463767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.466790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.488319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 849/1699] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:10.490099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.492526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:10.528840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca444c0-4749-44cc-8b27-cf57951d17be service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.550447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.551319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.553060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.553549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.566622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.574187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.574333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 851/1700] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:10 2022] PUT /identity/v3/projects/56c93f64004b42f6889cf348b9665306/users/01e0c9e04bb84f43bf8df91cdc2f592a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:33:10.574333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.581795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:10.583486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 850/1701] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:10.598682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:10.603581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:10.650135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:10.798284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94278b94-3a59-4c9f-a4c3-0c965aa9c56b service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.811948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598e015-fa4c-4631-8a6e-ebcb382de2fa admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:10.818610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.819411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.827997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.828464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] PATH_INFO: `/v3/projects/c39d18fcaa4740529a933f94a25244b3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.831707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c39d18fcaa4740529a933f94a25244b3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.832803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.925924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:10.927136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:10.927871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:10.928384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:10.930707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:10.975857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:10.978866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665b33a-00c2-4203-80d5-597b9b61fc11 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:10.983637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c39d18fcaa4740529a933f94a25244b3', 'request_id': 'req-849ee2dc-2d8f-4642-b045-8561dbc69fac'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.001075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 851/1702] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:11.003038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:11.004163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:11.005577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:11.061557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] MFA Rules not processed for user `01e0c9e04bb84f43bf8df91cdc2f592a`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:11.067203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:11.067666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:11.069110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c39d18fcaa4740529a933f94a25244b3', 'request_id': 'req-849ee2dc-2d8f-4642-b045-8561dbc69fac'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.081037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c39d18fcaa4740529a933f94a25244b3', 'request_id': 'req-849ee2dc-2d8f-4642-b045-8561dbc69fac'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.107771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] The token cache is being invalidate because project c39d18fcaa4740529a933f94a25244b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:11.109217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.110129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.114876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 852/1703] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:10 2022] DELETE /identity/v3/projects/c39d18fcaa4740529a933f94a25244b3 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:11.122700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:11.131045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:11.379935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 852/1704] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:11.382758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:11.387360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:11.408266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ee2dc-2d8f-4642-b045-8561dbc69fac admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:11.426643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:11.427488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:11.427918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:11.429527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] PATH_INFO: `/v3/users/54121084facd4ea488f90037cb8dc1f0` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:11.440898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=54121084facd4ea488f90037cb8dc1f0)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:11.443136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:11.474871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0caf12c-ed8d-40d5-827a-5646b45de651 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:11.480628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:11.499489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:11.501255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:11.501634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:11.505228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:11.506326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:11.516343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:11.658041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 853/1705] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:11 2022] POST /identity/v3/projects => generated 373 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:11.669929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:11.769473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54121084facd4ea488f90037cb8dc1f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.777931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54121084facd4ea488f90037cb8dc1f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.831676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54121084facd4ea488f90037cb8dc1f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.853324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54121084facd4ea488f90037cb8dc1f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.949937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95161f04-7084-4bee-b4c2-b9f6a74659c0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:11.965078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:11.968774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e0aeead-34bb-4db1-93fa-3d61479056cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54121084facd4ea488f90037cb8dc1f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:11.980242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:11.980691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 853/1706] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:33:11 2022] DELETE /identity/v3/users/54121084facd4ea488f90037cb8dc1f0 => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:11.981324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:11.983149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:11.984168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:11.986300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc333a-7bea-4f01-b10f-d2b9d56ae9a8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:11.991517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc333a-7bea-4f01-b10f-d2b9d56ae9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:11.992140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc333a-7bea-4f01-b10f-d2b9d56ae9a8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:11.996894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:12.007234 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 854/1707] 10.222.0.13 () {54 vars in 912 bytes} [Sat Sep 3 01:33:11 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:33:12.013207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:12.013743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:12.014182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:12.090145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:12.094083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:12.152819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 854/1708] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:11 2022] POST /identity/v3/users => generated 293 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:33:12.157075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:12.159621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:12.189655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e216b7-52e8-485f-aafc-958ca21cbe9a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:12.294801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 855/1709] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:33:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:12.467439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:12.468907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:12.469413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:12.470124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:12.471612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:12.473503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:12.501438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 855/1710] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:12.506226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:12.533847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e129f-5e04-4e2b-a94a-4d384c2914df None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:12.568088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:12.583335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:12.583826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:12.583826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] PATH_INFO: `/v3/projects/ea92ffb371544a3297a3bc7e45b6a71e/users/68c1eeb972fb441f8e06c97273b6cb05/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:12.662044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea92ffb371544a3297a3bc7e45b6a71e, user_id=68c1eeb972fb441f8e06c97273b6cb05, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:12.665414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:12.777010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 856/1711] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:12 2022] PUT /identity/v3/projects/ea92ffb371544a3297a3bc7e45b6a71e/users/68c1eeb972fb441f8e06c97273b6cb05/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:12.778731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:12.808624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73038259-4427-405c-ae12-2a230c3abdad admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:13.118817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ae6dd-bb5b-4987-80b5-ef28a2e7e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:13.119710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ae6dd-bb5b-4987-80b5-ef28a2e7e1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:13.120130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ae6dd-bb5b-4987-80b5-ef28a2e7e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:13.120696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ae6dd-bb5b-4987-80b5-ef28a2e7e1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:13.121586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ae6dd-bb5b-4987-80b5-ef28a2e7e1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:13.126394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ae6dd-bb5b-4987-80b5-ef28a2e7e1ef admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:13.167060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:13.167523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 856/1712] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:13.177484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ca76f-a0bf-4da1-95a5-b29c3ca3b929 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:13.196988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:13.197698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:13.197980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:13.198423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] PATH_INFO: `/v3/projects/ea92ffb371544a3297a3bc7e45b6a71e/users/68c1eeb972fb441f8e06c97273b6cb05/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:13.203433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea92ffb371544a3297a3bc7e45b6a71e, user_id=68c1eeb972fb441f8e06c97273b6cb05, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:13.207132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:13.252452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 857/1713] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:13 2022] PUT /identity/v3/projects/ea92ffb371544a3297a3bc7e45b6a71e/users/68c1eeb972fb441f8e06c97273b6cb05/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:13.264070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:13.264754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:13.265255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:13.293779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:13.293779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:13.340051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:13.346736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] MFA Rules not processed for user `68c1eeb972fb441f8e06c97273b6cb05`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:13.348114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:13.412257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:13.458328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa6952-5829-4277-846d-73eec4e3761a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:13.464824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:13.465803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:13.466326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:13.466773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:13.480445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:13.483182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:13.513254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 858/1714] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:13.574424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 857/1715] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:13.583794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:13.586094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:13.707860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6616bf-1370-4962-a411-dd47a8442cf9 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:13.720891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:13.723041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:13.725116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:13.726864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:13.764117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:13.765713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:13.813354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 859/1716] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:14.234092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:14.248095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29020dcd-4509-47e2-94b5-a8a76685b3fe None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:14.260176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:14.260997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:14.262487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:14.262930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:14.286238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:14.400378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:14.527601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:14.530465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c582d-4a29-4b57-9945-024696afa935 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:14.535359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:14.536794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:14.559938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 858/1717] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:14.714555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b107-86c2-453f-ad3c-2888bd03f3c6 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:14.738438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:14.738438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:14.738438 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:14.739494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] PATH_INFO: `/v3/projects/7b8e9de1bb5c48c29ac9a4f20c8e0afd` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:14.743025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b8e9de1bb5c48c29ac9a4f20c8e0afd)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:14.745191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:14.923236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d30fb-1427-44d1-8b4f-dc1785e321f0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:14.923803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d30fb-1427-44d1-8b4f-dc1785e321f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:14.924279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d30fb-1427-44d1-8b4f-dc1785e321f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:14.973994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b8e9de1bb5c48c29ac9a4f20c8e0afd', 'request_id': 'req-81661649-6432-4213-80bf-974e0f0e11f1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:15.005585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-318d30fb-1427-44d1-8b4f-dc1785e321f0 None None] MFA Rules not processed for user `409c324dca974d9dad9529b2a0d82b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:15.011614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-318d30fb-1427-44d1-8b4f-dc1785e321f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:15.065927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b8e9de1bb5c48c29ac9a4f20c8e0afd', 'request_id': 'req-81661649-6432-4213-80bf-974e0f0e11f1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:15.087109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b8e9de1bb5c48c29ac9a4f20c8e0afd', 'request_id': 'req-81661649-6432-4213-80bf-974e0f0e11f1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:15.149128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] The token cache is being invalidate because project 7b8e9de1bb5c48c29ac9a4f20c8e0afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:15.149434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:15.154191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-81661649-6432-4213-80bf-974e0f0e11f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:15.167069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 860/1718] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:14 2022] DELETE /identity/v3/projects/7b8e9de1bb5c48c29ac9a4f20c8e0afd => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:15.168561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:15.168852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:15.169139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:15.307268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:15.313680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:15.327949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 859/1719] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:15.353486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:15.593357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 861/1720] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:15.604156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:15.604445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:15.604721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:15.662992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:15.672143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:15.690693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 860/1721] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:15.705203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:15.722710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a10c58-d960-4a0e-8030-f0d946925fe0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:15.728640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:15.729612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:15.730294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:15.730882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:15.731946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:33:15.732391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:15.734273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:15.753528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 862/1722] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:33:15.759002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:15.777900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b2297-c390-46fd-aac3-e5976c8832f9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:15.784603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:15.785401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:15.785691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:15.785969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:15.786716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:15.789050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:16.038473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 861/1723] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:15 2022] POST /identity/v3/projects => generated 373 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:16.044125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:16.058525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8ac4f-572e-45dc-ae97-d40dbdb60d36 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:16.105673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:16.154643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:16.155776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.156204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.156618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.158199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:16.171606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:16.292560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 863/1724] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:16 2022] POST /identity/v3/users => generated 287 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:16.303658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:16.323163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030e92b-cfec-4a00-a81d-0e0d4404e149 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:16.336845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:16.340018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.341889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.342756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.346126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:16.351051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:16.395773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:16.419412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 862/1725] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:16.438159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ffaed-9626-45c6-a9e8-fb54f0317263 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:16.463582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853145c7-209f-43c6-9a2d-849af852ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:16.464045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853145c7-209f-43c6-9a2d-849af852ca24 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.465780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853145c7-209f-43c6-9a2d-849af852ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.466122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853145c7-209f-43c6-9a2d-849af852ca24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.466916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853145c7-209f-43c6-9a2d-849af852ca24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:16.469267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853145c7-209f-43c6-9a2d-849af852ca24 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:16.499628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 864/1726] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:16.500782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:16.527927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3256fc-1494-4b94-8002-435c8b3e73a0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:16.547967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f2f1d-4035-45f9-9b89-aac4c7c246f3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.553892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d191ff-0ca7-465d-857a-75c9a398344e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:16.553994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f2f1d-4035-45f9-9b89-aac4c7c246f3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.553994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f2f1d-4035-45f9-9b89-aac4c7c246f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.554120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d191ff-0ca7-465d-857a-75c9a398344e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.554120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d191ff-0ca7-465d-857a-75c9a398344e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.554381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d191ff-0ca7-465d-857a-75c9a398344e admin admin] PATH_INFO: `/v3/projects/27b46987345b460593f8cad9df427c6d/users/83e064518f7d422ab25b87ce93cc7320/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.583623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d191ff-0ca7-465d-857a-75c9a398344e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b46987345b460593f8cad9df427c6d, user_id=83e064518f7d422ab25b87ce93cc7320, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:16.587464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d191ff-0ca7-465d-857a-75c9a398344e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:16.590519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-032f2f1d-4035-45f9-9b89-aac4c7c246f3 None None] MFA Rules not processed for user `29b4f0836bbf4684bdc6177e42b88b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:16.593493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-032f2f1d-4035-45f9-9b89-aac4c7c246f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:16.720839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 863/1727] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:16 2022] PUT /identity/v3/projects/27b46987345b460593f8cad9df427c6d/users/83e064518f7d422ab25b87ce93cc7320/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:16.731640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395d4d-9e2c-42ae-957f-437e25ab0de7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.732290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395d4d-9e2c-42ae-957f-437e25ab0de7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.732978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26395d4d-9e2c-42ae-957f-437e25ab0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.837924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-26395d4d-9e2c-42ae-957f-437e25ab0de7 None None] MFA Rules not processed for user `83e064518f7d422ab25b87ce93cc7320`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:16.839427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-26395d4d-9e2c-42ae-957f-437e25ab0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:16.869631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 865/1728] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:16.871107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:16.871497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:16.871960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:16.922091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] MFA Rules not processed for user `2b9ab1983bb84318b509878b930e9402`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:16.927500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:17.095511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 864/1729] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:17.097007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:17.097306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:17.097682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:17.139800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 866/1730] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:17.263305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:17.265358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:17.396314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:17.407946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 865/1731] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:33:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:17.529111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:17.535419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:17.756583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0126d2-ad82-46af-ad9b-e782454f6242 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:17.775310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e18951-93da-4537-ac50-84ba7f166520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:17.776199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e18951-93da-4537-ac50-84ba7f166520 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:17.776716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e18951-93da-4537-ac50-84ba7f166520 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:17.777197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e18951-93da-4537-ac50-84ba7f166520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:17.806545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e18951-93da-4537-ac50-84ba7f166520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:17.808448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e18951-93da-4537-ac50-84ba7f166520 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:17.850726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 867/1732] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:18.004727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:18.006600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:18.302296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ec767-ec5e-4362-a9ee-1153ec32f731 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:18.309201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbe222-7087-476c-87e1-6b6a4441b688 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:18.309836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbe222-7087-476c-87e1-6b6a4441b688 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:18.310161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbe222-7087-476c-87e1-6b6a4441b688 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:18.318015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbe222-7087-476c-87e1-6b6a4441b688 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:18.321340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbe222-7087-476c-87e1-6b6a4441b688 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:18.323596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbe222-7087-476c-87e1-6b6a4441b688 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:18.340912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 866/1733] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:33:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:33:18.368945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:18.667199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:18.667487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:18.668010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:18.715632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] MFA Rules not processed for user `ecf076dde874415aa734644f27a10b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:18.719797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:18.904211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 868/1734] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:20.162745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:20.163757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:20.164793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:20.187995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:20.189309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:20.268198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 867/1735] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Sep 3 01:33:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:20.820019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:20.823036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:20.865168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d531bb8-6089-4636-96b7-6f5be1a8e00b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:20.899253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4989a-f6db-43ad-b287-7824d08e3af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:20.906716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4989a-f6db-43ad-b287-7824d08e3af8 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:20.912500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4989a-f6db-43ad-b287-7824d08e3af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:20.921347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4989a-f6db-43ad-b287-7824d08e3af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:20.957717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4989a-f6db-43ad-b287-7824d08e3af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:20.990601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4989a-f6db-43ad-b287-7824d08e3af8 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:21.054121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 869/1736] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:21.381459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:21.931186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:21.933243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:22.148526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03f4b2-4c90-40b4-8950-aee8416101f4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:22.185278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:22.186143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:22.186971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:22.187658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] PATH_INFO: `/v3/users/455815f31a444487af7bdd7ef8ae81fc` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:22.191956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=455815f31a444487af7bdd7ef8ae81fc)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:22.195358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:22.313805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:22.536035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455815f31a444487af7bdd7ef8ae81fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:22.659721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455815f31a444487af7bdd7ef8ae81fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:22.661376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455815f31a444487af7bdd7ef8ae81fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:22.684101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455815f31a444487af7bdd7ef8ae81fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:22.691256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455815f31a444487af7bdd7ef8ae81fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:22.698186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 868/1737] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:33:21 2022] DELETE /identity/v3/users/455815f31a444487af7bdd7ef8ae81fc => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:22.702661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4a81-610b-4e1e-bbc8-a66b1fa5072b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:22.703153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4a81-610b-4e1e-bbc8-a66b1fa5072b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:22.705287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b4a81-610b-4e1e-bbc8-a66b1fa5072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:22.741018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4a2b4a81-610b-4e1e-bbc8-a66b1fa5072b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:22.743012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4a2b4a81-610b-4e1e-bbc8-a66b1fa5072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:22.890910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 870/1738] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:22.902146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:22.903871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:22.998521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59d04-d7da-470a-9900-f2e39172bf9c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:23.004698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0abf5-e6d2-4f1a-b686-6811c8f74a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:23.005402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0abf5-e6d2-4f1a-b686-6811c8f74a3a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:23.010763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0abf5-e6d2-4f1a-b686-6811c8f74a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:23.010763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0abf5-e6d2-4f1a-b686-6811c8f74a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:23.017889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0abf5-e6d2-4f1a-b686-6811c8f74a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:23.019483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0abf5-e6d2-4f1a-b686-6811c8f74a3a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:23.033685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 869/1739] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:23.384460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02862265-b8de-4bf2-8867-472299e0bb25 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:23.384754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02862265-b8de-4bf2-8867-472299e0bb25 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:23.385045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02862265-b8de-4bf2-8867-472299e0bb25 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:23.386803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 871/1740] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:33:23.408334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:23.408880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:23.409324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:23.445319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:23.446884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:23.574726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 870/1741] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:33:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:23.707639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02862265-b8de-4bf2-8867-472299e0bb25 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:23.710254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-02862265-b8de-4bf2-8867-472299e0bb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:23.749773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02862265-b8de-4bf2-8867-472299e0bb25 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:23.765437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:23.766330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:23.766892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:23.767328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:23.772133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:23.781229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:23.783813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:23.798423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 872/1742] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:23.857680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:23.859436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:23.903289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2cdf9-843b-4456-8ed1-8088ac61d60d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:23.954525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:23.956680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:23.962555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:23.963224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] PATH_INFO: `/v3/projects/2f837df8976a4f8dbccd2b34b341acf5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:23.966081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f837df8976a4f8dbccd2b34b341acf5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:23.968835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 03 01:33:23.968835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Sep 03 01:33:23.968835 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: Sep 03 01:33:23.974715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:24.162438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:24.197106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f837df8976a4f8dbccd2b34b341acf5', 'request_id': 'req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.282712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f837df8976a4f8dbccd2b34b341acf5', 'request_id': 'req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.293437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f837df8976a4f8dbccd2b34b341acf5', 'request_id': 'req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.322958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] The token cache is being invalidate because project 2f837df8976a4f8dbccd2b34b341acf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:24.323385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.325048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.329133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 871/1743] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:23 2022] DELETE /identity/v3/projects/2f837df8976a4f8dbccd2b34b341acf5 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:24.334857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:24.350749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4066f4-9a58-4666-a89e-ae5ffbcfa914 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:24.529943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:24.531020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:24.531306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:24.531767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] PATH_INFO: `/v3/users/955b0777314044df86e118ec12238620` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:24.533063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=955b0777314044df86e118ec12238620)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:24.535069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:24.687718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:24.687718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:24.709057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1ac0e-a4e5-47c1-bc0d-ca64e8a38ebb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:24.716386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:24.717715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:24.721514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:24.721948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] PATH_INFO: `/v3/users/83e064518f7d422ab25b87ce93cc7320` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:24.723604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e064518f7d422ab25b87ce93cc7320)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:24.731255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:24.771845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955b0777314044df86e118ec12238620'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.773945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955b0777314044df86e118ec12238620'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.845453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955b0777314044df86e118ec12238620'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:24.864398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955b0777314044df86e118ec12238620'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.001184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-932d92b5-fb43-477f-afa4-b0ca87e66118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955b0777314044df86e118ec12238620'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.041161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 873/1744] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:33:24 2022] DELETE /identity/v3/users/955b0777314044df86e118ec12238620 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:25.064119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e064518f7d422ab25b87ce93cc7320'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.208370 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e064518f7d422ab25b87ce93cc7320'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.208951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e064518f7d422ab25b87ce93cc7320'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.260236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e064518f7d422ab25b87ce93cc7320'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.288673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e064518f7d422ab25b87ce93cc7320'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:25.303429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 872/1745] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:33:24 2022] DELETE /identity/v3/users/83e064518f7d422ab25b87ce93cc7320 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:25.308439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:25.308806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:25.309148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:25.372846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:25.378315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:25.640066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 874/1746] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:25.654664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:25.713623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:25.716530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:25.944268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422e99-9434-4e07-80c6-8e97c886b414 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:25.954125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:25.954838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:25.955128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:25.955409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:25.975672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:25.975672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:25.990667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 873/1747] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:27.692928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:27.694477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:27.712334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f1af5-36aa-453f-a824-ce3f34d25646 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:27.717182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:27.718824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:27.719092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:27.719496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] PATH_INFO: `/v3/projects/8dca69797b90429aba416d0e51692a15` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:27.724803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dca69797b90429aba416d0e51692a15)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:27.727677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:27.824067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:27.826305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:27.849128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100f927-e1c5-4dd7-9c8c-68b9f939ae8a service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:27.862570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:27.863467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:27.864117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:27.864393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] PATH_INFO: `/v3/projects/27b46987345b460593f8cad9df427c6d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:27.866241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b46987345b460593f8cad9df427c6d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:27.870210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:27.896385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dca69797b90429aba416d0e51692a15', 'request_id': 'req-e5353121-0509-4422-91cb-bd0befd980aa'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:27.998471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dca69797b90429aba416d0e51692a15', 'request_id': 'req-e5353121-0509-4422-91cb-bd0befd980aa'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.000350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:28.012218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dca69797b90429aba416d0e51692a15', 'request_id': 'req-e5353121-0509-4422-91cb-bd0befd980aa'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.017034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:28.034129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b46987345b460593f8cad9df427c6d', 'request_id': 'req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.043866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] The token cache is being invalidate because project 8dca69797b90429aba416d0e51692a15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:28.047782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.049591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e5353121-0509-4422-91cb-bd0befd980aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.052827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 875/1748] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:27 2022] DELETE /identity/v3/projects/8dca69797b90429aba416d0e51692a15 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:28.060469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52862601-a9be-496c-8ca8-ee82d593c9c2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.066025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52862601-a9be-496c-8ca8-ee82d593c9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.066025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52862601-a9be-496c-8ca8-ee82d593c9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.092267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-52862601-a9be-496c-8ca8-ee82d593c9c2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:28.093760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-52862601-a9be-496c-8ca8-ee82d593c9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:28.161994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b46987345b460593f8cad9df427c6d', 'request_id': 'req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.171486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b46987345b460593f8cad9df427c6d', 'request_id': 'req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.212592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] The token cache is being invalidate because project 27b46987345b460593f8cad9df427c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:33:28.212876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.213730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b28558b6-4bd1-48d8-aa39-8d0fd1b5bfdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:33:28.215368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 874/1749] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:33:27 2022] DELETE /identity/v3/projects/27b46987345b460593f8cad9df427c6d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:33:28.230932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e998726-a836-4c85-bf29-3ced5ca3b199 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.231253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e998726-a836-4c85-bf29-3ced5ca3b199 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.231528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e998726-a836-4c85-bf29-3ced5ca3b199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.241572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 876/1750] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:28.251173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23726230-1aa4-41fa-b040-749166b48afa None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.252490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23726230-1aa4-41fa-b040-749166b48afa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.253211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23726230-1aa4-41fa-b040-749166b48afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.277208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-23726230-1aa4-41fa-b040-749166b48afa None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:28.278891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-23726230-1aa4-41fa-b040-749166b48afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:28.299789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9e998726-a836-4c85-bf29-3ced5ca3b199 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:28.301639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9e998726-a836-4c85-bf29-3ced5ca3b199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:28.325445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 877/1751] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:28.330152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23726230-1aa4-41fa-b040-749166b48afa None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:28.336909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 875/1752] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:28.355777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474af0a-62ba-4983-a7f1-10b569262225 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.356559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474af0a-62ba-4983-a7f1-10b569262225 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.356978 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474af0a-62ba-4983-a7f1-10b569262225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.417992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8474af0a-62ba-4983-a7f1-10b569262225 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:28.421668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8474af0a-62ba-4983-a7f1-10b569262225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:28.432288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23726230-1aa4-41fa-b040-749166b48afa None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:28.443409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:28.447642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 876/1753] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:33:28.447642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c51798-65a8-4279-892c-ff736575deaf None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.447642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c51798-65a8-4279-892c-ff736575deaf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.448787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.448787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.450105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.450440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:33:28.450751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:28.451487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c51798-65a8-4279-892c-ff736575deaf None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.453799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 877/1754] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:33:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:33:28.454507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af889-dd87-43a5-8fdd-cd2e35a8a9f9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:28.455452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c51798-65a8-4279-892c-ff736575deaf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:28.485673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c51798-65a8-4279-892c-ff736575deaf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:28.496333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 878/1755] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:33:28.497759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.499290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.501443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.510199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:28.511109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.511973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.512455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.514395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:33:28.514958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:28.516886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:28.531138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:28.534297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:28.550121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 878/1756] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:33:28.550121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:28.572041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4242c96-faa5-4769-80d9-f1fe9b48926a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:28.597508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:28.600753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.601048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.601448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.608806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:28.616218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:28.729271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 879/1757] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:28.730936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:28.755505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35363b7-00c7-4278-9098-ff91201ce07c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:28.768713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:28.770016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.770303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.770658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.771523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:28.774634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:28.798506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 879/1758] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/projects => generated 377 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:28.807264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:28.921871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1e57d-cdcf-4600-b1e5-c341eb0e26ee admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:28.944253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:28.946845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:28.947241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:28.947740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:28.950126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 880/1759] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/projects => generated 369 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:28.950233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:28.950309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:28.950869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:28.951665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:29.053885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 880/1760] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:28 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:29.053885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.080648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff02c99-fca2-4bd7-a26c-afe767e4cf00 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.159754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7801da6-5828-44de-9a6c-be463ec13a90 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.166666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.171753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.171753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.171753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.198638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.200454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.243139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 881/1761] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:29.248147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.283218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.284128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.284578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.284997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.285783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.288062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.384661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08160a-e742-4a3a-bae7-e103f8f61004 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.394562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.400340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 881/1762] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:29 2022] POST /identity/v3/users => generated 285 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:29.401166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.401602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.402006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.402954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.404891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.412637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.426577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7374525-6caa-436b-8164-0eb1b87d0529 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.440746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 882/1763] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:29.441381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.442185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.442744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.443260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.444193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.445539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.447857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.463397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 882/1764] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:29.467806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.483171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77da4f-869c-4f47-8c1f-94cc3d2de6a9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.483546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4de281-b002-42e6-a4d6-b02b3c0c6491 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.489620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.490592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.490799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.491219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.491596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.492588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.492701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.493150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.493804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.493969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.495930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.496468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.509306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 883/1765] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:29.512518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 883/1766] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:33:29.515549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.521793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.532714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb090f4-407e-429c-81de-d162de1a937f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.537391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d6ec4-86b8-4b0b-80d7-2ea6bf9de5b0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:29.540070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ed3e-d8a2-401d-9e50-b03d1882e39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.541476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ed3e-d8a2-401d-9e50-b03d1882e39f admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.542238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ed3e-d8a2-401d-9e50-b03d1882e39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.543077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ed3e-d8a2-401d-9e50-b03d1882e39f admin admin] PATH_INFO: `/v3/projects/76923d86417843508f6272c87192ec55/users/457b8c1209304ba28d37b5f6cc133b07/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.555010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b685502-eee8-4f99-b900-da9993c0d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:29.556200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685502-eee8-4f99-b900-da9993c0d43e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.556634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685502-eee8-4f99-b900-da9993c0d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.557058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b685502-eee8-4f99-b900-da9993c0d43e admin admin] PATH_INFO: `/v3/projects/ae01041111db4aefb2647f5b062d5ff3/users/0f4e4d46d46847c1af305dc028a8da6f/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.571021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76ed3e-d8a2-401d-9e50-b03d1882e39f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76923d86417843508f6272c87192ec55, user_id=457b8c1209304ba28d37b5f6cc133b07, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.575608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76ed3e-d8a2-401d-9e50-b03d1882e39f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.591906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b685502-eee8-4f99-b900-da9993c0d43e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae01041111db4aefb2647f5b062d5ff3, user_id=0f4e4d46d46847c1af305dc028a8da6f, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:29.596912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b685502-eee8-4f99-b900-da9993c0d43e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:29.677959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 884/1767] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:29 2022] PUT /identity/v3/projects/76923d86417843508f6272c87192ec55/users/457b8c1209304ba28d37b5f6cc133b07/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:33:29.679403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 884/1768] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:29 2022] PUT /identity/v3/projects/ae01041111db4aefb2647f5b062d5ff3/users/0f4e4d46d46847c1af305dc028a8da6f/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:29.696288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.696783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.697112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:29.697583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.697751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:29.698193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:29.735201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] MFA Rules not processed for user `0f4e4d46d46847c1af305dc028a8da6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:29.736511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:29.762214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] MFA Rules not processed for user `457b8c1209304ba28d37b5f6cc133b07`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:29.763837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:29.795070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:29.815791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:29.906404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 885/1769] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:29.917269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:29.918837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:29.965708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 885/1770] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:29.974560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:30.201457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea374cef-056a-4939-aa78-10973a926c4a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:30.244690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:30.247583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:30.266170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:30.267051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:30.267817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:30.275068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9608ed-e751-4ce9-8cc1-f1f98863f5a6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:30.285660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:30.310189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:30.318603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:30.323171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:30.323453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:30.374258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:30.383836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:30.414291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 886/1771] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:33:30.488710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 886/1772] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:33:29 2022] POST /identity/v3/projects => generated 377 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:30.489931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:30.509285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff4715-a9b3-4e46-a0c1-78d666a44284 service glance] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:30.719432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:30.720069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:30.720351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:30.722114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:30.725283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:30.731307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:30.850185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 887/1773] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:33:30 2022] POST /identity/v3/users => generated 295 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:33:30.853450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:30.901376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4aca2-f8f8-4dcf-af28-7f441fc117ba admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:30.930077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:30.934397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:30.944720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:30.947209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:30.947697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:30.948548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:30.953558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:30.969819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 887/1774] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:30.973604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:30.992260 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9d58e-d706-46b0-924d-074a59c83e6d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:31.005424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:31.006148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:31.006615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:31.006893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] PATH_INFO: `/v3/projects/0247c2411d52420fb0f531cb0c16339d/users/cd2257295a1d4c5c91419be4bd856b6f/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:31.041334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0247c2411d52420fb0f531cb0c16339d, user_id=cd2257295a1d4c5c91419be4bd856b6f, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:31.045186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:31.138640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 888/1775] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:30 2022] PUT /identity/v3/projects/0247c2411d52420fb0f531cb0c16339d/users/cd2257295a1d4c5c91419be4bd856b6f/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:33:31.142363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:31.174126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53645e59-f0e3-4c2a-aa99-710f7321a182 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:31.273098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:31.417376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e5493-de8f-4c48-a684-2e0cf941d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:31.417991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e5493-de8f-4c48-a684-2e0cf941d7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:31.418438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e5493-de8f-4c48-a684-2e0cf941d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:31.419092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e5493-de8f-4c48-a684-2e0cf941d7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:31.419271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e5493-de8f-4c48-a684-2e0cf941d7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:31.420929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e5493-de8f-4c48-a684-2e0cf941d7ca admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:31.467732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 888/1776] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:33:31.472719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:31.488166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e98e77-88cf-4760-8179-7e510af06926 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:31.502654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:31.502654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:31.502654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:31.502654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] PATH_INFO: `/v3/projects/0247c2411d52420fb0f531cb0c16339d/users/cd2257295a1d4c5c91419be4bd856b6f/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:31.509525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0247c2411d52420fb0f531cb0c16339d, user_id=cd2257295a1d4c5c91419be4bd856b6f, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:31.512439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:31.539568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 889/1777] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:33:31 2022] PUT /identity/v3/projects/0247c2411d52420fb0f531cb0c16339d/users/cd2257295a1d4c5c91419be4bd856b6f/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:33:31.555546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb8cdf-3d26-4e49-9e16-1c0c9f467edd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:31.555927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb8cdf-3d26-4e49-9e16-1c0c9f467edd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:31.556544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb8cdf-3d26-4e49-9e16-1c0c9f467edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:31.676004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ebeb8cdf-3d26-4e49-9e16-1c0c9f467edd None None] MFA Rules not processed for user `cd2257295a1d4c5c91419be4bd856b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:31.677933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ebeb8cdf-3d26-4e49-9e16-1c0c9f467edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:31.849124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 889/1778] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:33:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:31.877108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:31.880364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:31.975803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f37059-3e39-4e97-90c2-249759859a85 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:31.980972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:31.981655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:31.981938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:31.982216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:31.996115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:32.067257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:32.071752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:32.088181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 890/1779] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:33.123971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:33.124474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:33.124550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:33.189927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:33.191399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:33.319321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 890/1780] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:33:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:33.334755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:33.337257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:33.640215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e370c-023f-4b9c-8d1a-f3c273c7c754 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:33.646511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed3aae-3e9a-4905-8f82-ac71da5d7d15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:33.647434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed3aae-3e9a-4905-8f82-ac71da5d7d15 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:33.647979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed3aae-3e9a-4905-8f82-ac71da5d7d15 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:33.648993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed3aae-3e9a-4905-8f82-ac71da5d7d15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:33.668462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed3aae-3e9a-4905-8f82-ac71da5d7d15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:33.670133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed3aae-3e9a-4905-8f82-ac71da5d7d15 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:33.689210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 891/1781] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 01:33:39.142605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:39.144420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:39.157125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47badc7d-d9a6-4a6f-b5de-4bec27bacf00 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:39.160942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee20f6-c896-409b-8c68-2ac0865750e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:39.161575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee20f6-c896-409b-8c68-2ac0865750e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:39.161862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee20f6-c896-409b-8c68-2ac0865750e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:39.162174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee20f6-c896-409b-8c68-2ac0865750e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:39.177016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee20f6-c896-409b-8c68-2ac0865750e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:39.179692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee20f6-c896-409b-8c68-2ac0865750e1 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:39.192999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 891/1782] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:43.683530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=54 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:46.655056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bba3d-c295-435e-ae8e-aad467f8b2ec None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:46.655056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bba3d-c295-435e-ae8e-aad467f8b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:46.655056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bba3d-c295-435e-ae8e-aad467f8b2ec None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:46.656219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 892/1783] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:33:46.666305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:46.666824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:46.667078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:46.689128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:33:46.690431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:33:46.704465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:46.780785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 892/1784] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:33:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:33:47.085398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bba3d-c295-435e-ae8e-aad467f8b2ec None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:33:47.102727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bba3d-c295-435e-ae8e-aad467f8b2ec None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:33:47.112534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a3e55-efed-4f3b-802a-abeef572c2ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:33:47.113250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a3e55-efed-4f3b-802a-abeef572c2ed service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:33:47.113570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a3e55-efed-4f3b-802a-abeef572c2ed service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:33:47.113897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a3e55-efed-4f3b-802a-abeef572c2ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:33:47.123724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a3e55-efed-4f3b-802a-abeef572c2ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:33:47.125307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a3e55-efed-4f3b-802a-abeef572c2ed service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:33:47.135603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 893/1785] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:33:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:33:56.800239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=24 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:33:57.131343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:16.100204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:16.190904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa7e19-5ac2-4f13-a5e8-552f57ba199d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:16.195912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:16.196548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:16.196825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:16.197095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] PATH_INFO: `/v3/users/457b8c1209304ba28d37b5f6cc133b07` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:16.198231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] RBAC: Authorizing `identity:delete_user(user_id=457b8c1209304ba28d37b5f6cc133b07)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:16.200450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:16.481733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457b8c1209304ba28d37b5f6cc133b07'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:16.544341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457b8c1209304ba28d37b5f6cc133b07'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:16.545671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457b8c1209304ba28d37b5f6cc133b07'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:16.570030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457b8c1209304ba28d37b5f6cc133b07'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:16.578931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457b8c1209304ba28d37b5f6cc133b07'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:16.588390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 893/1786] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:16 2022] DELETE /identity/v3/users/457b8c1209304ba28d37b5f6cc133b07 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:16.593107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b945d-e325-4176-9581-19df234383fe None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:16.593677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b945d-e325-4176-9581-19df234383fe None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:16.593677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b945d-e325-4176-9581-19df234383fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:16.617140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fb7b945d-e325-4176-9581-19df234383fe None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:16.618460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb7b945d-e325-4176-9581-19df234383fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:16.695622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 894/1787] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:16.707859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:16.709953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:16.819686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadd348-3394-40b7-9e60-8c3fa33d295a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:16.829604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:16.830627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:16.831094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:16.831569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:16.843518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:16.845070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:16.856109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 894/1788] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:17.492432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b945d-e325-4176-9581-19df234383fe None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:17.493990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb7b945d-e325-4176-9581-19df234383fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:17.521787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b945d-e325-4176-9581-19df234383fe None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:17.531093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:17.536939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:17.536939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:17.536939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] PATH_INFO: `/v3/projects/76923d86417843508f6272c87192ec55` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:17.536939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76923d86417843508f6272c87192ec55)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:17.538614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:17.754114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76923d86417843508f6272c87192ec55', 'request_id': 'req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:17.835632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76923d86417843508f6272c87192ec55', 'request_id': 'req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:17.843574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76923d86417843508f6272c87192ec55', 'request_id': 'req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:17.870039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] The token cache is being invalidate because project 76923d86417843508f6272c87192ec55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:17.870651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:17.873413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:17.875092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 895/1789] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:17 2022] DELETE /identity/v3/projects/76923d86417843508f6272c87192ec55 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:17.879177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:17.898287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32724b4-12f7-4c80-a986-15f5508e7d3b service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:17.924402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:18.020176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:18.021714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:18.022216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:18.022990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] PATH_INFO: `/v3/users/cd2257295a1d4c5c91419be4bd856b6f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:18.024264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2257295a1d4c5c91419be4bd856b6f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:18.026509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:18.238225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2257295a1d4c5c91419be4bd856b6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.298142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2257295a1d4c5c91419be4bd856b6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.298875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2257295a1d4c5c91419be4bd856b6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.320137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2257295a1d4c5c91419be4bd856b6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.331015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2257295a1d4c5c91419be4bd856b6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.338577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 895/1790] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:17 2022] DELETE /identity/v3/users/cd2257295a1d4c5c91419be4bd856b6f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:18.454183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:18.455959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:18.473033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a6ef4-9d62-4eb5-be04-f8d73d0488a7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:18.558383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:18.559766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:18.560248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:18.560820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] PATH_INFO: `/v3/users/01e0c9e04bb84f43bf8df91cdc2f592a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:18.563318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e0c9e04bb84f43bf8df91cdc2f592a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:18.570726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:18.653808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:18.751194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e0c9e04bb84f43bf8df91cdc2f592a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.751876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e0c9e04bb84f43bf8df91cdc2f592a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.765071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e0c9e04bb84f43bf8df91cdc2f592a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.772213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e0c9e04bb84f43bf8df91cdc2f592a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.831963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:18.834145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:18.848712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91b85-db69-497b-872b-29ab42ed26c9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:18.856604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f02a3a4-3dbd-4692-be2c-f7cbcd319060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e0c9e04bb84f43bf8df91cdc2f592a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:18.856698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:18.857102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:18.857393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:18.857675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] PATH_INFO: `/v3/projects/0247c2411d52420fb0f531cb0c16339d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:18.858969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0247c2411d52420fb0f531cb0c16339d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:18.861019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 896/1791] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:18 2022] DELETE /identity/v3/users/01e0c9e04bb84f43bf8df91cdc2f592a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:18.862885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:18.866144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:18.866606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:18.866882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:18.891786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:18.893366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:18.979364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 897/1792] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:18.989599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:18.991765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:19.045808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0247c2411d52420fb0f531cb0c16339d', 'request_id': 'req-d87ec382-c320-48fe-9c8a-929426bd179d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.063345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f160c-9660-442c-a430-75a92492bf08 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.118264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.118906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.119186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.119454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.132315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:19.142756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0247c2411d52420fb0f531cb0c16339d', 'request_id': 'req-d87ec382-c320-48fe-9c8a-929426bd179d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.149979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0247c2411d52420fb0f531cb0c16339d', 'request_id': 'req-d87ec382-c320-48fe-9c8a-929426bd179d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.163581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] The token cache is being invalidate because project 0247c2411d52420fb0f531cb0c16339d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:19.164201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.165100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d87ec382-c320-48fe-9c8a-929426bd179d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.167043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 896/1793] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:18 2022] DELETE /identity/v3/projects/0247c2411d52420fb0f531cb0c16339d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:19.172635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a428a15-2eeb-4571-b7af-448703a44677 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.173292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a428a15-2eeb-4571-b7af-448703a44677 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.173560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a428a15-2eeb-4571-b7af-448703a44677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.190606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.192077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.196531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2a428a15-2eeb-4571-b7af-448703a44677 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:19.198749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2a428a15-2eeb-4571-b7af-448703a44677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:19.203132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 898/1794] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:19.207537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 897/1795] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:19.215437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.217938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.217938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.243645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:19.244947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:19.257468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 898/1796] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:19.258551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.260387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:19.274780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a95066-74f4-4436-96ba-901517b1bc18 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.280216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.282177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.282177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.282177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.282554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:19.282639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.284215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.298800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 899/1797] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:34:19.303968 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.314391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abafbaa-b69d-4a6b-9038-1dca18c4c145 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.319653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.320313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.320596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.321045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.321690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.323986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.335262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:19.423455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 899/1798] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:19 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:19.424720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.437665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6f34c-17ef-4aa8-acf8-74b87c128aee admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.510515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.511185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.511467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.511742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.512429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.514382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.634947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 900/1799] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:19 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:19.638986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.651375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be2cd6-9502-4a96-a73b-96d1260ffbe4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.652915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.654825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:19.656194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.657182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.657619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.658043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.658877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.660823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.672716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 900/1800] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:19.676774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.682325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8eacb-32a6-4897-89c9-94b5255b43c3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.687260 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.687903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.688185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.688456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] PATH_INFO: `/v3/projects/56c93f64004b42f6889cf348b9665306` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.689614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c93f64004b42f6889cf348b9665306)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.690438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93e6d9-fb56-42db-90bd-fdab4da67f67 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.691563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.696096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.696735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.697032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.697316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.698024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.700476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.730394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 901/1801] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:19.735193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.745613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa10582-4ad0-4f9f-9f06-23535659026b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:19.750555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd6b73-c864-48f1-940d-e0d2b04b031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:19.751325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd6b73-c864-48f1-940d-e0d2b04b031b admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.751700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd6b73-c864-48f1-940d-e0d2b04b031b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.752050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd6b73-c864-48f1-940d-e0d2b04b031b admin admin] PATH_INFO: `/v3/projects/eb3086d351ea4611870f3dfd6a35e3cb/users/f324633a968b42db88ec9bb92d16ef9d/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.769560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd6b73-c864-48f1-940d-e0d2b04b031b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3086d351ea4611870f3dfd6a35e3cb, user_id=f324633a968b42db88ec9bb92d16ef9d, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:19.773029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd6b73-c864-48f1-940d-e0d2b04b031b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:19.847356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 902/1802] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:19 2022] PUT /identity/v3/projects/eb3086d351ea4611870f3dfd6a35e3cb/users/f324633a968b42db88ec9bb92d16ef9d/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:19.859538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:19.859932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:19.860315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:19.864504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c93f64004b42f6889cf348b9665306', 'request_id': 'req-a46383ae-5448-4b24-a92a-1af905fe1e71'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.891310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] MFA Rules not processed for user `f324633a968b42db88ec9bb92d16ef9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:19.892467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:19.956204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 903/1803] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:19.961555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:19.972253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c93f64004b42f6889cf348b9665306', 'request_id': 'req-a46383ae-5448-4b24-a92a-1af905fe1e71'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.979534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c93f64004b42f6889cf348b9665306', 'request_id': 'req-a46383ae-5448-4b24-a92a-1af905fe1e71'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:19.998775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] The token cache is being invalidate because project 56c93f64004b42f6889cf348b9665306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:19.999122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.000627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.005232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 901/1804] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:19 2022] DELETE /identity/v3/projects/56c93f64004b42f6889cf348b9665306 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:20.006887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.008661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.022235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:20.041433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9d490-40cf-4c9f-a8ee-d8424f4fe154 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.046576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.047206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.047486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.047757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.049774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.050646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.086392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46383ae-5448-4b24-a92a-1af905fe1e71 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.091678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.092462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.092954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.093232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] PATH_INFO: `/v3/users/68c1eeb972fb441f8e06c97273b6cb05` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.094435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c1eeb972fb441f8e06c97273b6cb05)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.096429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.157548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 904/1805] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:19 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:20.160926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.163388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.183940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:20.226829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8085bbf-b5a3-498a-bc0c-665267573092 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.231937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.232356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.232639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.232912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.233618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.235603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.336098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c1eeb972fb441f8e06c97273b6cb05'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.336893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c1eeb972fb441f8e06c97273b6cb05'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.350006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c1eeb972fb441f8e06c97273b6cb05'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.354285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 905/1806] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:20 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:20.355900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.357869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c1eeb972fb441f8e06c97273b6cb05'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.367654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8df209-b14f-46e8-9c76-97053c06bb2d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.372995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.374160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.374160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.374160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.374814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.376471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.389538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 906/1807] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:20.393518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.406162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0afb14-e78b-4cfb-812d-32fb0bef1c1a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.410991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.411549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.412551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.412836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] PATH_INFO: `/v3/projects/484b8a76e2044e59b12b65cbea1f22a5/users/6679742f7ad34d229b3d49e2aaca6ed7/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.429308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] RBAC: Authorizing `identity:create_grant(project_id=484b8a76e2044e59b12b65cbea1f22a5, user_id=6679742f7ad34d229b3d49e2aaca6ed7, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.432068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.440794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c1eeb972fb441f8e06c97273b6cb05'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:20.447585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 902/1808] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:20 2022] DELETE /identity/v3/users/68c1eeb972fb441f8e06c97273b6cb05 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:20.504051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 907/1809] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:20 2022] PUT /identity/v3/projects/484b8a76e2044e59b12b65cbea1f22a5/users/6679742f7ad34d229b3d49e2aaca6ed7/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:20.508011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.509601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.576905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02639da1-7bd3-4e71-9a16-3c44a043395f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.610119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:20.634852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbc219-00ab-4dd7-8c6d-fefb46f7a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.635471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbc219-00ab-4dd7-8c6d-fefb46f7a813 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.635757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbc219-00ab-4dd7-8c6d-fefb46f7a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.636025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbc219-00ab-4dd7-8c6d-fefb46f7a813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.636684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbc219-00ab-4dd7-8c6d-fefb46f7a813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.638298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbc219-00ab-4dd7-8c6d-fefb46f7a813 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.650969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 903/1810] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:20.655263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.667146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d39aea-626a-4a83-86bc-ae602ca0245d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.673635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.674606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.675046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.675613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] PATH_INFO: `/v3/projects/484b8a76e2044e59b12b65cbea1f22a5/users/6679742f7ad34d229b3d49e2aaca6ed7/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.678682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484b8a76e2044e59b12b65cbea1f22a5, user_id=6679742f7ad34d229b3d49e2aaca6ed7, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.682707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.700733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 908/1811] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:20 2022] PUT /identity/v3/projects/484b8a76e2044e59b12b65cbea1f22a5/users/6679742f7ad34d229b3d49e2aaca6ed7/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:20.711801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.712176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.712517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.742446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] MFA Rules not processed for user `6679742f7ad34d229b3d49e2aaca6ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:20.748603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.815398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.817773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.822691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 904/1812] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:20.843552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:20.845312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.888374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec14b9b-e7d4-425a-b894-11b0503b89d0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.893165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.893807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.894093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.894389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] PATH_INFO: `/v3/projects/ea92ffb371544a3297a3bc7e45b6a71e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.895811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea92ffb371544a3297a3bc7e45b6a71e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.899253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:20.913659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b87d3-5c3f-4f6b-a79a-4c405d555e6c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:20.917846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:20.918805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:20.919186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:20.919557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:20.921904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:20.991102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:20.992698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09480c-e450-4ac0-a81e-25b94ae4d070 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.004039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 905/1813] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:21.018441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea92ffb371544a3297a3bc7e45b6a71e', 'request_id': 'req-0c68d033-c748-48e7-a752-6f141d4f6920'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:21.087818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea92ffb371544a3297a3bc7e45b6a71e', 'request_id': 'req-0c68d033-c748-48e7-a752-6f141d4f6920'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:21.093379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:21.095852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea92ffb371544a3297a3bc7e45b6a71e', 'request_id': 'req-0c68d033-c748-48e7-a752-6f141d4f6920'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:21.114646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] The token cache is being invalidate because project ea92ffb371544a3297a3bc7e45b6a71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:21.114987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:21.115967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0c68d033-c748-48e7-a752-6f141d4f6920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:21.121684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 909/1814] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:20 2022] DELETE /identity/v3/projects/ea92ffb371544a3297a3bc7e45b6a71e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:21.127978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.127978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.127978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.148819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:21.150049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:21.155985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:21.210553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 906/1815] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:21.218271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.218591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.218868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.241381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:21.243480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:21.254672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 910/1816] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:21.255994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:21.265908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eaccb-8868-4aef-9137-8cc62c9e1d43 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:21.270626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:21.271928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.272227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.272528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.273155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:21.273460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:21.275226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.288203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 907/1817] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:34:21.292596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:21.302815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011a7e-77f4-4f7b-a26e-dc99af521fa8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:21.307865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:21.308533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.308820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.309095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.309745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:21.311750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.425086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 911/1818] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:21 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:21.429198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:21.439581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2bf713-d2b9-4c02-9873-ce1142d9f336 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:21.512370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:21.512981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.513268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.513540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.514181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:21.516177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.721273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 908/1819] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:21 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:21.725714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:21.735928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a6961-03c1-459a-86c9-17fa3dd67e0f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:21.741621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:21.747189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.747189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.747189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.747189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:21.747189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.762227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 912/1820] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:21.766140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:21.779007 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413c047-5844-434d-a923-8b40a22f206e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:21.782820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504c07f-d7b8-420c-b9b0-775ee77a5398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:21.783423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504c07f-d7b8-420c-b9b0-775ee77a5398 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.783702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504c07f-d7b8-420c-b9b0-775ee77a5398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.783971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504c07f-d7b8-420c-b9b0-775ee77a5398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.784612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504c07f-d7b8-420c-b9b0-775ee77a5398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:21.786212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504c07f-d7b8-420c-b9b0-775ee77a5398 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.801377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 909/1821] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:21.804906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:21.817990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb92b92-10a9-4c6b-b65e-adb0888fa6bf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:21.834262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:21.835007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.835292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.835571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] PATH_INFO: `/v3/projects/3baa84a6684f493082f3991387ba1543/users/6e73ab52dbb44805b767a45a46599f8f/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.851461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3baa84a6684f493082f3991387ba1543, user_id=6e73ab52dbb44805b767a45a46599f8f, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:21.854429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:21.949551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 913/1822] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:21 2022] PUT /identity/v3/projects/3baa84a6684f493082f3991387ba1543/users/6e73ab52dbb44805b767a45a46599f8f/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:21.960604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:21.961065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:21.961177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:21.989745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] MFA Rules not processed for user `6e73ab52dbb44805b767a45a46599f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:21.990966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:22.063133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 910/1823] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:22.067682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.144354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc490b3-a9f7-4aa9-be32-d155f0ae4245 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:22.154165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:22.155025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.155730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.156231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.157079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:22.159833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:22.278240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 914/1824] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:22 2022] POST /identity/v3/projects => generated 361 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:22.282645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.301105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b5fac-ef45-425f-8c55-0dc15da8f7f8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:22.305467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:22.376317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:22.376947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.377231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.377507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.378158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:22.380155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:22.454555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 911/1825] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:22 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:22.458922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.470975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72c189-a7d9-4882-babb-9f40c62f68dc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:22.475749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:22.476742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.476742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.476973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.477637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:22.479356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:22.491190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 915/1826] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:22.498177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.509782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d686a-789a-4edf-8b14-94d5c39f24d9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:22.514510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:22.515192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.515473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.515745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] PATH_INFO: `/v3/projects/fbde2bdd10d3476dbe74de969bfea569/users/5e1ce201f2c042d1848bd900fef56477/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.532217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbde2bdd10d3476dbe74de969bfea569, user_id=5e1ce201f2c042d1848bd900fef56477, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:22.534920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:22.637350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 912/1827] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:22 2022] PUT /identity/v3/projects/fbde2bdd10d3476dbe74de969bfea569/users/5e1ce201f2c042d1848bd900fef56477/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:22.642319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.654082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5db59-938a-4d35-be20-f68dcc268416 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:22.720154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429f508-7fd6-4e61-bf80-31d9d3b900aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:22.720765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429f508-7fd6-4e61-bf80-31d9d3b900aa admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.721046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429f508-7fd6-4e61-bf80-31d9d3b900aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.721316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429f508-7fd6-4e61-bf80-31d9d3b900aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.721959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429f508-7fd6-4e61-bf80-31d9d3b900aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:22.723623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429f508-7fd6-4e61-bf80-31d9d3b900aa admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:22.724966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:22.734804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 916/1828] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:22.739305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.751791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b796cc-d786-4ffc-a9db-cf61b14d6288 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:22.758663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:22.758663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.758912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.759360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] PATH_INFO: `/v3/projects/fbde2bdd10d3476dbe74de969bfea569/users/5e1ce201f2c042d1848bd900fef56477/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.762209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbde2bdd10d3476dbe74de969bfea569, user_id=5e1ce201f2c042d1848bd900fef56477, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:22.765701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:22.779739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 913/1829] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:22 2022] PUT /identity/v3/projects/fbde2bdd10d3476dbe74de969bfea569/users/5e1ce201f2c042d1848bd900fef56477/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:22.792390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:22.792680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:22.792962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:22.831622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] MFA Rules not processed for user `5e1ce201f2c042d1848bd900fef56477`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:22.834204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:22.923717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 917/1830] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:22.937698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:22.940057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:23.026663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f45dc-3ecd-43c3-88ff-c77ab2b4069b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:23.030246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6793964-6a6f-464a-a3f2-4735d1b94036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:23.031150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6793964-6a6f-464a-a3f2-4735d1b94036 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:23.031431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6793964-6a6f-464a-a3f2-4735d1b94036 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:23.031703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6793964-6a6f-464a-a3f2-4735d1b94036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:23.041429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6793964-6a6f-464a-a3f2-4735d1b94036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:23.042916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6793964-6a6f-464a-a3f2-4735d1b94036 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:23.053564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 914/1831] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:23.944714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:24.010620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac237129-5b2d-4ecf-862c-11e25e5fac59 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:24.015517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:24.016159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:24.016481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:24.016752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] PATH_INFO: `/v3/users/6e73ab52dbb44805b767a45a46599f8f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:24.018457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e73ab52dbb44805b767a45a46599f8f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:24.020691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:24.184029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e73ab52dbb44805b767a45a46599f8f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.251888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e73ab52dbb44805b767a45a46599f8f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.252587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e73ab52dbb44805b767a45a46599f8f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.265399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e73ab52dbb44805b767a45a46599f8f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.273078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e73ab52dbb44805b767a45a46599f8f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.275337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:24.279648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 918/1832] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:23 2022] DELETE /identity/v3/users/6e73ab52dbb44805b767a45a46599f8f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:24.284758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:24.285317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:24.285516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:24.291429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:24.293000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:24.305518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:24.306760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:24.362701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebc57e-996e-47aa-94c8-51b28829115d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:24.368378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 915/1833] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:24.369267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:24.369923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:24.370208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:24.370522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] PATH_INFO: `/v3/users/f324633a968b42db88ec9bb92d16ef9d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:24.371689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f324633a968b42db88ec9bb92d16ef9d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:24.373636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:24.378065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:24.379719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:24.385640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:24.439035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a789a-ef8d-453f-b0b5-b821d9fda92b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:24.442596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ba532-054f-48ce-8c6b-84b064e438f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:24.443826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ba532-054f-48ce-8c6b-84b064e438f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:24.444086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ba532-054f-48ce-8c6b-84b064e438f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:24.444353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ba532-054f-48ce-8c6b-84b064e438f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:24.453101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ba532-054f-48ce-8c6b-84b064e438f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:24.454554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197ba532-054f-48ce-8c6b-84b064e438f4 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:24.464977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 916/1834] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:24.568765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f324633a968b42db88ec9bb92d16ef9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.652014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f324633a968b42db88ec9bb92d16ef9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.652905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f324633a968b42db88ec9bb92d16ef9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.670186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f324633a968b42db88ec9bb92d16ef9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.681225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f324633a968b42db88ec9bb92d16ef9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:24.712797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 919/1835] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:24 2022] DELETE /identity/v3/users/f324633a968b42db88ec9bb92d16ef9d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:24.713105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:24.713105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:24.713105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:24.742357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:24.744660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:24.829296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 917/1836] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:24.841513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:24.843482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:24.920394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc6401-9e56-4123-9c64-027050807558 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:24.923996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:24.924673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:24.924983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:24.925262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:24.939195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:24.942023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:24.959120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 920/1837] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:25.031871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:25.033587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:25.054542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea82ddd-997b-4308-a2e8-243dc1df6a27 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:25.059834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:25.068135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:25.068135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:25.068135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] PATH_INFO: `/v3/projects/3baa84a6684f493082f3991387ba1543` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:25.068135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3baa84a6684f493082f3991387ba1543)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:25.068135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:25.180555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3baa84a6684f493082f3991387ba1543', 'request_id': 'req-9672bf50-5155-4b83-8466-9eff5e734298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.226505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3baa84a6684f493082f3991387ba1543', 'request_id': 'req-9672bf50-5155-4b83-8466-9eff5e734298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.246410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3baa84a6684f493082f3991387ba1543', 'request_id': 'req-9672bf50-5155-4b83-8466-9eff5e734298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.259042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] The token cache is being invalidate because project 3baa84a6684f493082f3991387ba1543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:25.259408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.260169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.262198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 918/1838] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:25 2022] DELETE /identity/v3/projects/3baa84a6684f493082f3991387ba1543 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:25.267300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:25.281620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc99142-5589-4bee-b15c-ff10e2382312 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:25.302730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:25.345099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:25.347073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:25.347747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:25.347747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] PATH_INFO: `/v3/users/5e1ce201f2c042d1848bd900fef56477` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:25.349803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e1ce201f2c042d1848bd900fef56477)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:25.352007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:25.493309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:25.498446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:25.507996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672bf50-5155-4b83-8466-9eff5e734298 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:25.512595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:25.513429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:25.513558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:25.513829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] PATH_INFO: `/v3/projects/eb3086d351ea4611870f3dfd6a35e3cb` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:25.514973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3086d351ea4611870f3dfd6a35e3cb)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:25.516878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:25.545232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:25.556802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e1ce201f2c042d1848bd900fef56477'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.640650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3086d351ea4611870f3dfd6a35e3cb', 'request_id': 'req-553d09b5-c824-4751-a178-1bd04bbaf113'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.652060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e1ce201f2c042d1848bd900fef56477'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.652683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e1ce201f2c042d1848bd900fef56477'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.666276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e1ce201f2c042d1848bd900fef56477'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.672989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e1ce201f2c042d1848bd900fef56477'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.679294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 921/1839] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:25 2022] DELETE /identity/v3/users/5e1ce201f2c042d1848bd900fef56477 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:25.801142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3086d351ea4611870f3dfd6a35e3cb', 'request_id': 'req-553d09b5-c824-4751-a178-1bd04bbaf113'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.807461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3086d351ea4611870f3dfd6a35e3cb', 'request_id': 'req-553d09b5-c824-4751-a178-1bd04bbaf113'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.819059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] The token cache is being invalidate because project eb3086d351ea4611870f3dfd6a35e3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:25.819417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.820049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:25.821548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 919/1840] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:25 2022] DELETE /identity/v3/projects/eb3086d351ea4611870f3dfd6a35e3cb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:25.825809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:25.827263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:25.893987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9d606-3bf6-427e-91c2-4e20e0524c51 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:25.899772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:25.900409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:25.900696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:25.900974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] PATH_INFO: `/v3/users/6679742f7ad34d229b3d49e2aaca6ed7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:25.902117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6679742f7ad34d229b3d49e2aaca6ed7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:25.904111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:26.065679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:26.069371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6679742f7ad34d229b3d49e2aaca6ed7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.140196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:26.141858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:26.176375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6679742f7ad34d229b3d49e2aaca6ed7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.176375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6679742f7ad34d229b3d49e2aaca6ed7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.194191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6679742f7ad34d229b3d49e2aaca6ed7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.203841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8df07303-b296-4a8b-b4fe-a05b495e2af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6679742f7ad34d229b3d49e2aaca6ed7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.211537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 922/1841] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:25 2022] DELETE /identity/v3/users/6679742f7ad34d229b3d49e2aaca6ed7 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:26.217232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d09b5-c824-4751-a178-1bd04bbaf113 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:26.299378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:26.299779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.300065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.300418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] PATH_INFO: `/v3/projects/fbde2bdd10d3476dbe74de969bfea569` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.301954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbde2bdd10d3476dbe74de969bfea569)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:26.304824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:26.413485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbde2bdd10d3476dbe74de969bfea569', 'request_id': 'req-df636519-ba11-4fe5-92d8-162c3181511d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.480698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbde2bdd10d3476dbe74de969bfea569', 'request_id': 'req-df636519-ba11-4fe5-92d8-162c3181511d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.485230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:26.487849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbde2bdd10d3476dbe74de969bfea569', 'request_id': 'req-df636519-ba11-4fe5-92d8-162c3181511d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.500162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] The token cache is being invalidate because project fbde2bdd10d3476dbe74de969bfea569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:26.500871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.501173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-df636519-ba11-4fe5-92d8-162c3181511d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.502742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 920/1842] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:26 2022] DELETE /identity/v3/projects/fbde2bdd10d3476dbe74de969bfea569 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:26.508034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.508330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.508612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.529218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:26.530741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:26.593271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 923/1843] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:26.601373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.601658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.601936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.638515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:26.638515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:26.644489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 921/1844] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:26.649033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:26.659527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba2350-4e30-4be9-8228-344a8238a1fd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:26.664883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:26.665552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.666593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.666593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.666803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:26.667105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:26.668714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:26.680953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 924/1845] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:34:26.685200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:26.696271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f55707-b1be-4a81-a75c-479a02884b8e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:26.697151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:26.699082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:26.701076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:26.701568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.701846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.702110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.702794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:26.704712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:26.723272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a784a2b-7eb2-4cb2-8d93-8a66b19ef06c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:26.725876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:26.726546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.726827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.727123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] PATH_INFO: `/v3/projects/484b8a76e2044e59b12b65cbea1f22a5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.728239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=484b8a76e2044e59b12b65cbea1f22a5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:26.730353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:26.796836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 922/1846] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:26 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:26.801068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:26.843666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484b8a76e2044e59b12b65cbea1f22a5', 'request_id': 'req-b39c0409-f028-4165-862e-939310d495fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.869366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507f68-ab15-4083-a656-aba212bed92f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:26.925621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:26.926237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.926566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.926835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.927485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:26.929487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:26.931456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484b8a76e2044e59b12b65cbea1f22a5', 'request_id': 'req-b39c0409-f028-4165-862e-939310d495fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.937706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484b8a76e2044e59b12b65cbea1f22a5', 'request_id': 'req-b39c0409-f028-4165-862e-939310d495fc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.950516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] The token cache is being invalidate because project 484b8a76e2044e59b12b65cbea1f22a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:26.950806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.951482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b39c0409-f028-4165-862e-939310d495fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:26.953010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 925/1847] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:26 2022] DELETE /identity/v3/projects/484b8a76e2044e59b12b65cbea1f22a5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:26.958041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f06616-4e21-45a2-90d6-33da3ff2a660 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:26.959281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f06616-4e21-45a2-90d6-33da3ff2a660 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:26.959684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f06616-4e21-45a2-90d6-33da3ff2a660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:26.985127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-00f06616-4e21-45a2-90d6-33da3ff2a660 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:26.986362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-00f06616-4e21-45a2-90d6-33da3ff2a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:26.995035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 926/1848] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:27.003099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.003383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.003681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.019941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 923/1849] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:26 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:27.024194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:27.025208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.025411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:27.026809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:27.033664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 927/1850] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:27.038014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.039705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e15034-cf66-4b9d-a919-ce5a5680df6f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.044587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.045222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.045503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.045773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.046475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.047105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2684468-e2dc-4e73-b0b6-7b1e6a1e2cff None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.048062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.051722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.052879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.053151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.053417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.054060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:27.054382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.055970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.059158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 924/1851] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:27.063261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.068533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 928/1852] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:34:27.076138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db252f-1495-4737-86c7-2f9c59674733 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.077111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.079175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.079778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.080060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.080332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.080975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.082628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.083227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:27.087227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc802c6b-65d7-40ea-b86f-df05971fc192 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.092029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.119542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.119542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.119542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.120012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 925/1853] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:27.120168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.120168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.123001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.134269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9a83f-bfab-40c2-aea1-9313aca6613f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.139316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9acba4-c07d-4b65-8db1-b92a36615068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.142488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9acba4-c07d-4b65-8db1-b92a36615068 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.142488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9acba4-c07d-4b65-8db1-b92a36615068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.142488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9acba4-c07d-4b65-8db1-b92a36615068 admin admin] PATH_INFO: `/v3/projects/57009fc3d1f045599ee227142b46c5b8/users/e6e446f51f1349a9bde81260355f9a20/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.157027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9acba4-c07d-4b65-8db1-b92a36615068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57009fc3d1f045599ee227142b46c5b8, user_id=e6e446f51f1349a9bde81260355f9a20, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.159873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9acba4-c07d-4b65-8db1-b92a36615068 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.219887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 929/1854] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:27.224610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.241419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 926/1855] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:27 2022] PUT /identity/v3/projects/57009fc3d1f045599ee227142b46c5b8/users/e6e446f51f1349a9bde81260355f9a20/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:27.255664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.255951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.256225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.295704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869bf2d-0d38-4ab5-b9e4-093329f6c014 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.303250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:27.311222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] MFA Rules not processed for user `e6e446f51f1349a9bde81260355f9a20`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:27.312569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:27.359799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.360820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.361553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.362042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.363264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.365749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.391146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 927/1856] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:27.395335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.406974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92d2ef-fa4c-4113-90e3-21a60a64f401 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.415207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.415860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.416158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.416450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.417258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.419765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.467937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 930/1857] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:27.472107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.485563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f23d8c-1c25-4a1a-8c63-540a62817357 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.491421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.492162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.492444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.492713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.493396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.500811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.506483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 931/1858] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:27.510422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.525467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53aedee-56a3-4231-9f39-159ef5b63b1f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.530382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.531401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.531691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.531970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.532506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 928/1859] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:27.533103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.534364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.536233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.546561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 932/1860] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:27.555990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.566953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8aadfa-b22c-4ac8-b6b3-280bc378689f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.606268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18239777-ded6-41c2-b8cc-3491017d5994 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.610770 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614ba71-524c-456f-88a5-72cc13648c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.611428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614ba71-524c-456f-88a5-72cc13648c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.611711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614ba71-524c-456f-88a5-72cc13648c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.612154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614ba71-524c-456f-88a5-72cc13648c73 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.612855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614ba71-524c-456f-88a5-72cc13648c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.615043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614ba71-524c-456f-88a5-72cc13648c73 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.623985 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:27.631872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.632535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.632864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.633091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] PATH_INFO: `/v3/projects/efd244ba290249568e3a6f8bd3e7e814/users/205b8f60af7a4957a578f21f9ea464dd/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.649949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd244ba290249568e3a6f8bd3e7e814, user_id=205b8f60af7a4957a578f21f9ea464dd, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.652257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.750108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 933/1861] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:27 2022] PUT /identity/v3/projects/efd244ba290249568e3a6f8bd3e7e814/users/205b8f60af7a4957a578f21f9ea464dd/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:27.750598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 929/1862] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:27.754417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.757605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.759797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.759797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.779968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:27.781617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:27.843204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85db0-85b6-48cb-bf52-642010cbdfaf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.849149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.849869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.850159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.850512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.851170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.852997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.867137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 934/1863] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:27.871036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.875645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 930/1864] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:27.883228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3959-7d10-49e9-b376-fec160cbafbd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.883475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:27.885084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:27.888601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.889365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.889727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.890077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] PATH_INFO: `/v3/projects/e2dc19dcfeaa43999d585c933696d40f/users/de4e571872bb46f096556a319432ab3b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.900869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:27.907426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2dc19dcfeaa43999d585c933696d40f, user_id=de4e571872bb46f096556a319432ab3b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.910390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.959877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf8e21-2642-4e15-bb89-700f9a7a9c5d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:27.963613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:27.964223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:27.964500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:27.964777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:27.974135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:27.975612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:27.984405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 935/1865] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:27 2022] PUT /identity/v3/projects/e2dc19dcfeaa43999d585c933696d40f/users/de4e571872bb46f096556a319432ab3b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:27.986963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 931/1866] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:27.988526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:28.052948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d0baf6-d769-4371-9132-559b21ca63ff admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:28.058357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac559ea-255c-45e5-bf05-c40f4c8dcf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:28.059001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac559ea-255c-45e5-bf05-c40f4c8dcf67 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:28.059282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac559ea-255c-45e5-bf05-c40f4c8dcf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:28.059558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac559ea-255c-45e5-bf05-c40f4c8dcf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:28.060337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac559ea-255c-45e5-bf05-c40f4c8dcf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:28.061962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac559ea-255c-45e5-bf05-c40f4c8dcf67 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:28.079403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 936/1867] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:28.080912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:28.093390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cb24e-abb4-4f42-a1e8-b3c07b8bab73 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:28.099822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:28.100817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:28.101171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:28.102266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] PATH_INFO: `/v3/projects/e2dc19dcfeaa43999d585c933696d40f/users/de4e571872bb46f096556a319432ab3b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:28.103934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2dc19dcfeaa43999d585c933696d40f, user_id=de4e571872bb46f096556a319432ab3b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:28.108386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:28.134432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 932/1868] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:28 2022] PUT /identity/v3/projects/e2dc19dcfeaa43999d585c933696d40f/users/de4e571872bb46f096556a319432ab3b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:28.140767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d482ff-8b91-4393-990f-5d6c1f4f2c23 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:28.141058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d482ff-8b91-4393-990f-5d6c1f4f2c23 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:28.141340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d482ff-8b91-4393-990f-5d6c1f4f2c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:28.193820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-05d482ff-8b91-4393-990f-5d6c1f4f2c23 None None] MFA Rules not processed for user `de4e571872bb46f096556a319432ab3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:28.195104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-05d482ff-8b91-4393-990f-5d6c1f4f2c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:28.258638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:28.295916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 937/1869] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:28.307633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:28.309965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:28.418842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ca13a-4814-4b5a-889e-9f31353eb911 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:28.424944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:28.425771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:28.426928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:28.427216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:28.438065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:28.441655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:28.450266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 933/1870] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:29.408279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:29.408846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:29.408937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:29.444604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] MFA Rules not processed for user `205b8f60af7a4957a578f21f9ea464dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:29.446094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:29.524677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 938/1871] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:29.535967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:29.546224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648c3b8-016a-4e82-ac81-72cc09062633 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:29.550050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421efce-0e79-4479-8d0a-a1aa37e97a57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:29.550681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421efce-0e79-4479-8d0a-a1aa37e97a57 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:29.550962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421efce-0e79-4479-8d0a-a1aa37e97a57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:29.551231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421efce-0e79-4479-8d0a-a1aa37e97a57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:29.559919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421efce-0e79-4479-8d0a-a1aa37e97a57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:29.561345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421efce-0e79-4479-8d0a-a1aa37e97a57 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:29.570556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 934/1872] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:31.279908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:31.368696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3a5b9-5b99-4eb2-8f8a-207429701958 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:31.377324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:31.378230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:31.378647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:31.379022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] PATH_INFO: `/v3/users/e6e446f51f1349a9bde81260355f9a20` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:31.380344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e446f51f1349a9bde81260355f9a20)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:31.384304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:31.407531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22909793-0939-4626-97f7-66ebf2da7eeb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:31.407999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22909793-0939-4626-97f7-66ebf2da7eeb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:31.408183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22909793-0939-4626-97f7-66ebf2da7eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:31.430405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-22909793-0939-4626-97f7-66ebf2da7eeb None None] MFA Rules not processed for user `205b8f60af7a4957a578f21f9ea464dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:31.431637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-22909793-0939-4626-97f7-66ebf2da7eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:31.445444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 935/1873] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:31.564257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e446f51f1349a9bde81260355f9a20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:31.663894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e446f51f1349a9bde81260355f9a20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:31.665084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e446f51f1349a9bde81260355f9a20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:31.675495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:31.681672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e446f51f1349a9bde81260355f9a20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:31.688649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e446f51f1349a9bde81260355f9a20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:31.697335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 939/1874] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:31 2022] DELETE /identity/v3/users/e6e446f51f1349a9bde81260355f9a20 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:31.701825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:31.702256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:31.702985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:31.723365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:31.725410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:31.807595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 936/1875] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:31.819081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:31.820514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:31.905277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a6553-0603-4816-974f-51efcf73c9ac admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:31.912383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:31.913401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:31.913866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:31.914606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:31.925832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:31.928143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:31.941231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 940/1876] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:32.483230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:32.484711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:32.500429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f197e2-f1a5-4580-a3f2-fa5415ea3115 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:32.506835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:32.511212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:32.511212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:32.511212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] PATH_INFO: `/v3/projects/57009fc3d1f045599ee227142b46c5b8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:32.511212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] RBAC: Authorizing `identity:delete_project(project_id=57009fc3d1f045599ee227142b46c5b8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:32.511212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:32.529866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:32.612694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57009fc3d1f045599ee227142b46c5b8', 'request_id': 'req-578a039d-a32a-4d51-8132-9be4502c756c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:32.657640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57009fc3d1f045599ee227142b46c5b8', 'request_id': 'req-578a039d-a32a-4d51-8132-9be4502c756c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:32.664134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57009fc3d1f045599ee227142b46c5b8', 'request_id': 'req-578a039d-a32a-4d51-8132-9be4502c756c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:32.676526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] The token cache is being invalidate because project 57009fc3d1f045599ee227142b46c5b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:32.676820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:32.677488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:32.679839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 937/1877] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:32 2022] DELETE /identity/v3/projects/57009fc3d1f045599ee227142b46c5b8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:32.683998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:32.693555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd1fef-ebd9-4523-a854-16a057c3952e service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:32.749746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:32.750416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:32.750699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:32.750974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] PATH_INFO: `/v3/users/de4e571872bb46f096556a319432ab3b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:32.752096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4e571872bb46f096556a319432ab3b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:32.754036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:32.933045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4e571872bb46f096556a319432ab3b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:33.052059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4e571872bb46f096556a319432ab3b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:33.052767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4e571872bb46f096556a319432ab3b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:33.065415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4e571872bb46f096556a319432ab3b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:33.073320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81675861-4f60-478f-86eb-88d6ff183ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4e571872bb46f096556a319432ab3b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:33.079836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 941/1878] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:32 2022] DELETE /identity/v3/users/de4e571872bb46f096556a319432ab3b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:33.794772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:33.796347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:33.822757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a039d-a32a-4d51-8132-9be4502c756c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:33.891272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:33.891938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:33.892218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:33.892489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] PATH_INFO: `/v3/projects/e2dc19dcfeaa43999d585c933696d40f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:33.893644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2dc19dcfeaa43999d585c933696d40f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:33.895958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.032737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2dc19dcfeaa43999d585c933696d40f', 'request_id': 'req-6db57dad-3e4f-4400-9bbf-b6415729fe49'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:34.098083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2dc19dcfeaa43999d585c933696d40f', 'request_id': 'req-6db57dad-3e4f-4400-9bbf-b6415729fe49'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:34.105146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2dc19dcfeaa43999d585c933696d40f', 'request_id': 'req-6db57dad-3e4f-4400-9bbf-b6415729fe49'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:34.117139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] The token cache is being invalidate because project e2dc19dcfeaa43999d585c933696d40f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:34.117427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:34.118069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6db57dad-3e4f-4400-9bbf-b6415729fe49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:34.119613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 938/1879] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:33 2022] DELETE /identity/v3/projects/e2dc19dcfeaa43999d585c933696d40f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:34.124721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.125188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.125265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.134269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:34.145442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:34.146747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:34.230538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 942/1880] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:34.233881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.234215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.234604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.261633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:34.263431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:34.307386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 939/1881] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:34.309828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:34.325859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64cbea-5e2f-4634-aff2-61bbd0024e91 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:34.330941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:34.331605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.331885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.332160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.332826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:34.333148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:34.335533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.352383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 943/1882] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:34:34.354351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:34.368987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dabde0-6f35-4058-b32c-603d6357aa2a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:34.374591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:34.375345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.375723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.376089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.377344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:34.378994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.491244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 940/1883] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:34 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:34.492566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:34.518009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b9163-09a5-4197-8bb8-2aed5f6fb08b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:34.588855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:34.589507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.589762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.590033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.590703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:34.592663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.679178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 944/1884] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:34 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:34.683394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:34.696592 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3b740-5bca-432c-b88c-2496981a89d4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:34.701398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:34.702018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.702716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.702716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.703299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:34.704942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.706690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:34.718015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 941/1885] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:34.722263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:34.734531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880ae8a-9262-4f23-940c-82a650224511 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:34.740255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2096c7-d5fc-4c2f-90b4-7908580edadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:34.741354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2096c7-d5fc-4c2f-90b4-7908580edadb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.741728 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2096c7-d5fc-4c2f-90b4-7908580edadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.742073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2096c7-d5fc-4c2f-90b4-7908580edadb admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.742865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2096c7-d5fc-4c2f-90b4-7908580edadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:34.744705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2096c7-d5fc-4c2f-90b4-7908580edadb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.758442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 945/1886] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:34.765709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:34.776818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57575c8-86ec-4f82-9176-0fd678bb9dd1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:34.783399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:34.784280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.784660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.784939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] PATH_INFO: `/v3/projects/ab95254539c8429c8c58202918b62bfe/users/93c8cd928e3d4e5ebb500782565ed8a5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.809184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab95254539c8429c8c58202918b62bfe, user_id=93c8cd928e3d4e5ebb500782565ed8a5, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:34.812050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:34.875511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 942/1887] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:34 2022] PUT /identity/v3/projects/ab95254539c8429c8c58202918b62bfe/users/93c8cd928e3d4e5ebb500782565ed8a5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:34.887783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:34.888075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:34.888446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:34.937334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] MFA Rules not processed for user `93c8cd928e3d4e5ebb500782565ed8a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:34.939081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:35.043427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:35.053863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 946/1888] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:35.057652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:35.174879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d043e-3da0-4e20-9b47-fa2852bd047a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:35.181697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:35.183132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.184080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.184548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.185508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:35.188414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:35.290091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 943/1889] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:35 2022] POST /identity/v3/projects => generated 385 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:35.291249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:35.304764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5ca8a-ece4-44ef-a839-44323de61cd7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:35.395534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:35.396747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.397274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.397747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.398608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:35.402469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:35.478537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 947/1890] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:35 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:35.482242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:35.493779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef235bb-05ea-4f4a-8810-3a329d3ee6ed admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:35.498942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:35.499655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.500217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.500481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.501242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:35.503003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:35.515704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 944/1891] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:35.519782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:35.531342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6719253-e758-47e0-8d1a-9ef572e0f343 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:35.536153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:35.536890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.537334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.537603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] PATH_INFO: `/v3/projects/cbf4b17ad85840a59f5aa7e9d1a7fb60/users/ec4dc699d76247528843213a6ab8332f/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.555452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf4b17ad85840a59f5aa7e9d1a7fb60, user_id=ec4dc699d76247528843213a6ab8332f, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:35.558559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:35.705303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 948/1892] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:35 2022] PUT /identity/v3/projects/cbf4b17ad85840a59f5aa7e9d1a7fb60/users/ec4dc699d76247528843213a6ab8332f/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:35.706607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:35.725408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25189b99-e622-465e-9554-3b2893d49187 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:35.794925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600cc39-0d31-408e-8679-b61e8ebd1e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:35.795694 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600cc39-0d31-408e-8679-b61e8ebd1e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.795979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600cc39-0d31-408e-8679-b61e8ebd1e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.796241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600cc39-0d31-408e-8679-b61e8ebd1e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.796972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600cc39-0d31-408e-8679-b61e8ebd1e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:35.804518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600cc39-0d31-408e-8679-b61e8ebd1e9c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:35.817263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 945/1893] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:35.822906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:35.835614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae607b05-4351-4c07-97a5-abcdcb31875c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:35.841507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:35.842722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.843082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.843436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] PATH_INFO: `/v3/projects/cbf4b17ad85840a59f5aa7e9d1a7fb60/users/ec4dc699d76247528843213a6ab8332f/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.845798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf4b17ad85840a59f5aa7e9d1a7fb60, user_id=ec4dc699d76247528843213a6ab8332f, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:35.849022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:35.863971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 949/1894] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:35 2022] PUT /identity/v3/projects/cbf4b17ad85840a59f5aa7e9d1a7fb60/users/ec4dc699d76247528843213a6ab8332f/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:35.874794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:35.875154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:35.875556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:35.923095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] MFA Rules not processed for user `ec4dc699d76247528843213a6ab8332f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:35.924813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:35.962858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:36.004444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 946/1895] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:36.017035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:36.018905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:36.082306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf77732-7ad5-4aae-8e11-04993432e1bd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:36.087562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabeecc-8b47-400a-8e94-f62fd92d80d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:36.088258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabeecc-8b47-400a-8e94-f62fd92d80d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:36.088647 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabeecc-8b47-400a-8e94-f62fd92d80d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:36.089089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabeecc-8b47-400a-8e94-f62fd92d80d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:36.099157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabeecc-8b47-400a-8e94-f62fd92d80d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:36.101096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabeecc-8b47-400a-8e94-f62fd92d80d9 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:36.111049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 950/1896] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:36.144447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:36.235381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bcbf1-a9e0-49b4-bedf-613205b9fa74 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:36.240990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:36.242095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:36.242424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:36.242710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] PATH_INFO: `/v3/users/93c8cd928e3d4e5ebb500782565ed8a5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:36.244180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c8cd928e3d4e5ebb500782565ed8a5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:36.247170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:36.608699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c8cd928e3d4e5ebb500782565ed8a5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:36.609297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c8cd928e3d4e5ebb500782565ed8a5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:36.621568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c8cd928e3d4e5ebb500782565ed8a5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:36.628504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c8cd928e3d4e5ebb500782565ed8a5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:36.760718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c8cd928e3d4e5ebb500782565ed8a5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:36.768470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 947/1897] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:36 2022] DELETE /identity/v3/users/93c8cd928e3d4e5ebb500782565ed8a5 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:36.775268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:36.775268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:36.775268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:36.809784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:36.809784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:36.861235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:36.872689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 951/1898] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:36.883090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:36.884679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:36.968251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:36.975553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4f4c3-8c2f-45a9-be6e-a601823bae56 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:36.980050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:36.980584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:36.981096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:36.982143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:36.995991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:36.997584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:37.010433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 948/1899] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:37.718923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:37.720435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:37.733888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad1c30-91fe-4511-98c4-6da5a7ed36fd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:37.740531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:37.741680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:37.741970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:37.742310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] PATH_INFO: `/v3/projects/ab95254539c8429c8c58202918b62bfe` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:37.743876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab95254539c8429c8c58202918b62bfe)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:37.768489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:37.952731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab95254539c8429c8c58202918b62bfe', 'request_id': 'req-b332714c-1981-41af-bfab-4aa8a2904d3d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.017767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab95254539c8429c8c58202918b62bfe', 'request_id': 'req-b332714c-1981-41af-bfab-4aa8a2904d3d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.026196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab95254539c8429c8c58202918b62bfe', 'request_id': 'req-b332714c-1981-41af-bfab-4aa8a2904d3d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.046008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] The token cache is being invalidate because project ab95254539c8429c8c58202918b62bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:38.046008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.046008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b332714c-1981-41af-bfab-4aa8a2904d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.047827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 952/1900] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:37 2022] DELETE /identity/v3/projects/ab95254539c8429c8c58202918b62bfe => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:38.051876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:38.065021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66c6b-ab07-4057-acf7-92f09a6fcd22 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:38.186360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:38.187271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:38.188232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:38.190698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] PATH_INFO: `/v3/users/ec4dc699d76247528843213a6ab8332f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:38.195557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4dc699d76247528843213a6ab8332f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:38.197536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:38.437050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4dc699d76247528843213a6ab8332f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.437892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4dc699d76247528843213a6ab8332f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.472103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4dc699d76247528843213a6ab8332f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.481021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4dc699d76247528843213a6ab8332f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.538313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-73c44903-ef54-4255-a7b3-34cded967423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4dc699d76247528843213a6ab8332f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:38.546096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 949/1901] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:38 2022] DELETE /identity/v3/users/ec4dc699d76247528843213a6ab8332f => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:38.697970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4692f0-1fcf-40a1-921b-70536d965eb2 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:38.698294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4692f0-1fcf-40a1-921b-70536d965eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:38.699082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4692f0-1fcf-40a1-921b-70536d965eb2 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:38.704464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 953/1902] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:34:38.721730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:38.722018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:38.722370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:38.757004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:38.758432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:38.863854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 950/1903] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:34:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:38.884206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4692f0-1fcf-40a1-921b-70536d965eb2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:38.886132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9c4692f0-1fcf-40a1-921b-70536d965eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:38.888612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:38.890107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:38.901146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:38.923305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4692f0-1fcf-40a1-921b-70536d965eb2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:38.931340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:38.931972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:38.932489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:38.932679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:38.945233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:38.947229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:38.968753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 954/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:39.016478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de3385-4074-45a2-b088-cfe6b93752f5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.022423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.023314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.023729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.024230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] PATH_INFO: `/v3/users/d73c9c5108a845fdbd25b67bcb3a00f4` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.025581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73c9c5108a845fdbd25b67bcb3a00f4)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.027668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.168097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.170887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.186832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c982d-c291-4654-86da-5e9d166f1261 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.201034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.201690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.201972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.202242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] PATH_INFO: `/v3/projects/cbf4b17ad85840a59f5aa7e9d1a7fb60` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.203430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf4b17ad85840a59f5aa7e9d1a7fb60)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.205428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.236814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73c9c5108a845fdbd25b67bcb3a00f4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.238605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73c9c5108a845fdbd25b67bcb3a00f4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.254065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73c9c5108a845fdbd25b67bcb3a00f4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.261688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73c9c5108a845fdbd25b67bcb3a00f4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.320435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf4b17ad85840a59f5aa7e9d1a7fb60', 'request_id': 'req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.327068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f941337f-eeb1-4481-a1b1-bb11a93d3601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73c9c5108a845fdbd25b67bcb3a00f4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.334697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 951/1905] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:38 2022] DELETE /identity/v3/users/d73c9c5108a845fdbd25b67bcb3a00f4 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:39.391215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf4b17ad85840a59f5aa7e9d1a7fb60', 'request_id': 'req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.399679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf4b17ad85840a59f5aa7e9d1a7fb60', 'request_id': 'req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.414226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] The token cache is being invalidate because project cbf4b17ad85840a59f5aa7e9d1a7fb60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:39.414553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.415213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bd2975c-df32-4de4-aa4f-4dfcb3aa89f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.435542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 955/1906] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:39 2022] DELETE /identity/v3/projects/cbf4b17ad85840a59f5aa7e9d1a7fb60 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:39.435698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.435698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.435698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.446531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:39.447855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.511830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 952/1907] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:39.519482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.519766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.520040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.540638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:39.541893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.549876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 956/1908] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:39.554989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.564812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83c738-7e88-4567-919c-5af108724e5a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.567854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.569424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.570184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.570819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.571099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.571370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.572025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:39.573775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.583597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.599876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c553f-496d-4515-852c-39b83ca282f8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.605120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 953/1909] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:34:39.605266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.605365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.605613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.605882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] PATH_INFO: `/v3/projects/c0a44ea5393c4fb58fce35e8cbc4103b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.606373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.607039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a44ea5393c4fb58fce35e8cbc4103b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.608950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.620347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86229530-849e-4185-ba83-e97a6e625da4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.626861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.627465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.627745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.628008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.628646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.630632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.642458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:39.698695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:39.705068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a44ea5393c4fb58fce35e8cbc4103b', 'request_id': 'req-fb443239-03f1-4c62-9696-81a0f52bb674'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.718148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 954/1910] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:39.722176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.784822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1838018-d80c-4a77-adb5-38bd131d4c52 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.790270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.790904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.791183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.791453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.792076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.794057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.823447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a44ea5393c4fb58fce35e8cbc4103b', 'request_id': 'req-fb443239-03f1-4c62-9696-81a0f52bb674'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.829502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a44ea5393c4fb58fce35e8cbc4103b', 'request_id': 'req-fb443239-03f1-4c62-9696-81a0f52bb674'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.840939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] The token cache is being invalidate because project c0a44ea5393c4fb58fce35e8cbc4103b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:39.841232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.841870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fb443239-03f1-4c62-9696-81a0f52bb674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:39.843392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 957/1911] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:39 2022] DELETE /identity/v3/projects/c0a44ea5393c4fb58fce35e8cbc4103b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:39.848105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6787-ee99-4d49-9ace-569b67d9a013 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.848475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6787-ee99-4d49-9ace-569b67d9a013 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.848833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb6787-ee99-4d49-9ace-569b67d9a013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.869168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ecdb6787-ee99-4d49-9ace-569b67d9a013 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:39.870408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ecdb6787-ee99-4d49-9ace-569b67d9a013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.878096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 958/1912] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:39.885744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.886028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.886301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.907682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 955/1913] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:39.908955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:39.909856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.911796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.911946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:39.921359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 959/1914] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:39.926950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.936807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b665968-7612-498c-a7f5-ca725d4e8d39 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.937011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90c5ca-fb25-4ec8-8334-c9cbc118718a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.941423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.941602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.942027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.942151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.942305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.942452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.942616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.942732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.943413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.943533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:39.943767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.945115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.945357 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.958320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 956/1915] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:39.959664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 960/1916] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:34:39.960339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.963874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:39.970686 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f31a-d60f-42de-9856-30dd943322c2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.975849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2281a5-d8e5-49ad-94b5-3b6ea3697e68 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:39.976674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.977305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.977700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.977975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.978678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.979967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:39.980276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.980579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:39.980863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:39.981202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:39.981844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:39.983831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:39.991182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 957/1917] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:39.995319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.005246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e4b6f-8d9a-4ec9-a08a-1df269066546 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.010186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8abec-1bd5-4e3b-b437-fba75dc2932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.010899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8abec-1bd5-4e3b-b437-fba75dc2932a admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.011177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8abec-1bd5-4e3b-b437-fba75dc2932a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.011454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8abec-1bd5-4e3b-b437-fba75dc2932a admin admin] PATH_INFO: `/v3/projects/a1c281b554994cd4ad29c7ae1b9bee3c/users/17e30b1fe9594626bf2ceb6a6213c849/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.027963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8abec-1bd5-4e3b-b437-fba75dc2932a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c281b554994cd4ad29c7ae1b9bee3c, user_id=17e30b1fe9594626bf2ceb6a6213c849, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.030905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8abec-1bd5-4e3b-b437-fba75dc2932a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.113852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 961/1918] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:39 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:40.115040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.122032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 958/1919] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:39 2022] PUT /identity/v3/projects/a1c281b554994cd4ad29c7ae1b9bee3c/users/17e30b1fe9594626bf2ceb6a6213c849/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:40.135807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.136092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.136368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.169171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] MFA Rules not processed for user `17e30b1fe9594626bf2ceb6a6213c849`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:40.170554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:40.190284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965fccf-1990-4e57-b2c5-9a2693f87e5b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.245025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 959/1920] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:40.249826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.251437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.252064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.252350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.252630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.253269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.255259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.260710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2491a-a0bb-425a-a1c4-4b66f3dcc2ff None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.269960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.269960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.269960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.269960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.270430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.270430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.355984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 962/1921] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:40 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:40.360748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.371941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ccc18-3d50-435c-9c08-8f5410dbc2cd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.376601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.377227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.377512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.377817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.378538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.380150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.390550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:40.391031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 963/1922] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:40.401055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.402786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 960/1923] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:40 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:40.403772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.408834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:40.414732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a86fb-5e1c-430d-bce3-312d631d6807 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.471252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353e2fc-4215-4d7f-ad6d-469c21bddca1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.476769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.477570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.478152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.478638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.478950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.479250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.479530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.479945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.480595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.480962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.482228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.483441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.496296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 964/1924] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:40.499426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.509196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b39a0-82c9-456a-9c7d-cf56f2a9b1f6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.513934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72362d-c020-416e-9b19-bbcb78f40d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.514625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72362d-c020-416e-9b19-bbcb78f40d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.514905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72362d-c020-416e-9b19-bbcb78f40d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.515177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72362d-c020-416e-9b19-bbcb78f40d93 admin admin] PATH_INFO: `/v3/projects/7ef3228c9b934185bce8b64f3415e6fd/users/f50f82989dab48158c16e92f1b1efa7a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.530315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72362d-c020-416e-9b19-bbcb78f40d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef3228c9b934185bce8b64f3415e6fd, user_id=f50f82989dab48158c16e92f1b1efa7a, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.533249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72362d-c020-416e-9b19-bbcb78f40d93 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.608198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 961/1925] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:40 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:40.612207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.625778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e4af9-bdc3-489f-b9eb-8ec4eb5b3344 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.631434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 965/1926] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:40 2022] PUT /identity/v3/projects/7ef3228c9b934185bce8b64f3415e6fd/users/f50f82989dab48158c16e92f1b1efa7a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:34:40.642314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.642644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.643003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.678484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] MFA Rules not processed for user `f50f82989dab48158c16e92f1b1efa7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:40.679828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:40.704410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.705143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.705430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.705704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.706443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.708150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.720351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 962/1927] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:40.726436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.738584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25d939-94ed-423c-b4dc-321e33a4ae18 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.744224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.744919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.745198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.745480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] PATH_INFO: `/v3/projects/056637fad5484bd581357ce644d2e299/users/cbfa24e5a4ab4f4f99c63dd109ba7db7/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.753845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 966/1928] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:40.763355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056637fad5484bd581357ce644d2e299, user_id=cbfa24e5a4ab4f4f99c63dd109ba7db7, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.766119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.766889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.767587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:40.854641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 963/1929] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:40 2022] PUT /identity/v3/projects/056637fad5484bd581357ce644d2e299/users/cbfa24e5a4ab4f4f99c63dd109ba7db7/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:40.858615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.863100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6335e-0bba-4deb-ba73-a03d6504229b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.890085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:40.932293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.932878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.933166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.933437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.938332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1c0d5-5cb9-4362-b891-7df332a98e64 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.944522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.945368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.945993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.946274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.946969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.948885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:40.951431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:40.961777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 964/1930] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:40.967960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:40.978604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60e308-7556-4710-b408-2c6ff3bd9d0d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:40.983863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df39955-1cf2-46fe-afa0-37829521a6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:40.985987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df39955-1cf2-46fe-afa0-37829521a6a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:40.985987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df39955-1cf2-46fe-afa0-37829521a6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:40.985987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df39955-1cf2-46fe-afa0-37829521a6a1 admin admin] PATH_INFO: `/v3/projects/056637fad5484bd581357ce644d2e299/users/cbfa24e5a4ab4f4f99c63dd109ba7db7/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:40.989692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df39955-1cf2-46fe-afa0-37829521a6a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056637fad5484bd581357ce644d2e299, user_id=cbfa24e5a4ab4f4f99c63dd109ba7db7, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:40.992950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df39955-1cf2-46fe-afa0-37829521a6a1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:41.004773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 965/1931] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:40 2022] PUT /identity/v3/projects/056637fad5484bd581357ce644d2e299/users/cbfa24e5a4ab4f4f99c63dd109ba7db7/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:41.008772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:41.013437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:41.019322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:41.019569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:41.020072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:41.020599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 967/1932] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:34:41.055104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] MFA Rules not processed for user `cbfa24e5a4ab4f4f99c63dd109ba7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:41.056062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:41.141885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 966/1933] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:41.157952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:41.159786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:41.173682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba813-f2bf-48f2-ba48-8166e576845b service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:41.230421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4287a17-20c4-47f0-936b-93bc940e8925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:41.230954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4287a17-20c4-47f0-936b-93bc940e8925 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:41.231231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4287a17-20c4-47f0-936b-93bc940e8925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:41.231498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4287a17-20c4-47f0-936b-93bc940e8925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:41.241604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4287a17-20c4-47f0-936b-93bc940e8925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:41.243955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4287a17-20c4-47f0-936b-93bc940e8925 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:41.267186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 968/1934] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:42.414627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:42.480617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98af0e3-0a12-43c6-b7c0-080961146d9e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:42.485173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:42.485818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:42.486100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:42.486400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] PATH_INFO: `/v3/users/17e30b1fe9594626bf2ceb6a6213c849` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:42.487538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e30b1fe9594626bf2ceb6a6213c849)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:42.489521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:42.712547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e30b1fe9594626bf2ceb6a6213c849'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:42.713242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e30b1fe9594626bf2ceb6a6213c849'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:42.726350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e30b1fe9594626bf2ceb6a6213c849'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:42.731692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:42.733660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e30b1fe9594626bf2ceb6a6213c849'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:42.812113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e30b1fe9594626bf2ceb6a6213c849'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:42.818504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 967/1935] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:42 2022] DELETE /identity/v3/users/17e30b1fe9594626bf2ceb6a6213c849 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:42.823041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:42.831542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:42.831542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:42.843067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:42.844453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:42.910077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 969/1936] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:42.917938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:42.919534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:42.994424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c1e8d-e885-4a10-b1d9-30c66fb2235a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:42.998558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:43.004981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:43.004981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:43.004981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:43.011018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:43.014719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:43.030507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 968/1937] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:43.510810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:43.512320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:43.528155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3c98e-6fd4-449d-96a6-4052e8000e0f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:43.534646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:43.535747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:43.536206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:43.536530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] PATH_INFO: `/v3/projects/a1c281b554994cd4ad29c7ae1b9bee3c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:43.537929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c281b554994cd4ad29c7ae1b9bee3c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:43.549564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:43.568251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:43.681881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c281b554994cd4ad29c7ae1b9bee3c', 'request_id': 'req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:43.738770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c281b554994cd4ad29c7ae1b9bee3c', 'request_id': 'req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:43.745590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c281b554994cd4ad29c7ae1b9bee3c', 'request_id': 'req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:43.763425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] The token cache is being invalidate because project a1c281b554994cd4ad29c7ae1b9bee3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:43.763631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:43.764290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:43.766241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 970/1938] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:43 2022] DELETE /identity/v3/projects/a1c281b554994cd4ad29c7ae1b9bee3c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:43.773475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:43.781870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2447-c9e9-4693-91df-609ccc6824c3 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:43.849850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:43.850554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:43.850833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:43.851102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] PATH_INFO: `/v3/users/cbfa24e5a4ab4f4f99c63dd109ba7db7` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:43.852242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfa24e5a4ab4f4f99c63dd109ba7db7)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:43.854249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:44.041754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfa24e5a4ab4f4f99c63dd109ba7db7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.042454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfa24e5a4ab4f4f99c63dd109ba7db7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.056780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfa24e5a4ab4f4f99c63dd109ba7db7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.064251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfa24e5a4ab4f4f99c63dd109ba7db7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.149096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d86de51a-11a3-41cc-8eb4-5e673715778e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfa24e5a4ab4f4f99c63dd109ba7db7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.156092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 969/1939] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:43 2022] DELETE /identity/v3/users/cbfa24e5a4ab4f4f99c63dd109ba7db7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:44.657236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:44.658856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:44.671925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4e8e-b7b9-42d4-bd62-3d7ad0a6a8a7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:44.745892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:44.746687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:44.747039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:44.747450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] PATH_INFO: `/v3/projects/056637fad5484bd581357ce644d2e299` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:44.748696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=056637fad5484bd581357ce644d2e299)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:44.750686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:44.860126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056637fad5484bd581357ce644d2e299', 'request_id': 'req-dea331e1-11aa-44e3-b78c-266b38715b2e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.937865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056637fad5484bd581357ce644d2e299', 'request_id': 'req-dea331e1-11aa-44e3-b78c-266b38715b2e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.949062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056637fad5484bd581357ce644d2e299', 'request_id': 'req-dea331e1-11aa-44e3-b78c-266b38715b2e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.962678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] The token cache is being invalidate because project 056637fad5484bd581357ce644d2e299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:44.962974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.968403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dea331e1-11aa-44e3-b78c-266b38715b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:44.968403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 971/1940] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:44 2022] DELETE /identity/v3/projects/056637fad5484bd581357ce644d2e299 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:44.970984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:44.971494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:44.971743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:44.992492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:44.993718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:44.999071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:45.054167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 970/1941] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:45.062858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.063050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.063342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.088704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:45.089919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:45.097857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 972/1942] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:45.102111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.112065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c74699-7dd5-4805-9e4b-30d785a5229d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.117942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.118608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.118892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.119194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.119849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:45.120152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.121829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.138517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 971/1943] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:34:45.141331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.152681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf264f-37f7-4e7d-8f63-15e9503c7bd9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.157669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.158283 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.158629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.158972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.159611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.161547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.266402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 973/1944] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:45 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:45.267765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.279194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fe1bd-a7ae-4575-a533-e6c999386917 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.354616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.355277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.355558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.355830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.356496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.358422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.441482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 972/1945] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:45 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:45.445951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.451669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:45.456914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571901-7f0e-4bd4-bdb9-651b36287535 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.462241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.464620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.464620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.464620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.464620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.465656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.476652 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 974/1946] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:45.480697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.491634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06ac3d-d67b-4589-b6f0-8b59f5a7c082 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.496233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4320d-d0ea-4659-81a5-5b9b45f284ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.496842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4320d-d0ea-4659-81a5-5b9b45f284ee admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.497121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4320d-d0ea-4659-81a5-5b9b45f284ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.497390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4320d-d0ea-4659-81a5-5b9b45f284ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.498033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4320d-d0ea-4659-81a5-5b9b45f284ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.499760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4320d-d0ea-4659-81a5-5b9b45f284ee admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.512242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 973/1947] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:45.516135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.526184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e79a3-5719-45a3-bfc1-7d65996d78cb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.531290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.532060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.532562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.533314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] PATH_INFO: `/v3/projects/3fe77c11f92542898c50ba656dfc15ba/users/fd74c9e834d2404d9ea054f2619f1ee9/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.550634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe77c11f92542898c50ba656dfc15ba, user_id=fd74c9e834d2404d9ea054f2619f1ee9, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.553970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.629902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 975/1948] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:45 2022] PUT /identity/v3/projects/3fe77c11f92542898c50ba656dfc15ba/users/fd74c9e834d2404d9ea054f2619f1ee9/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:45.641397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.641616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.641892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.671880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] MFA Rules not processed for user `fd74c9e834d2404d9ea054f2619f1ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:45.673108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:45.741462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 974/1949] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:45.746036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.815797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db2b65-33cb-4d3f-9d1f-321e59f81d9d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.821516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.822201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.822519 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.822791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.823438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.825356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:45.893214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 976/1950] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:45 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:45.897766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:45.908690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483f22d-871f-43d1-b34c-535bafc906d6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:45.912202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:45.968247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:45.968982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:45.969270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:45.969553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:45.970196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:45.972185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:46.049158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 975/1951] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:45 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:46.050624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:46.061079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1dd4d-551b-46a8-b506-cc3bc40de37e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:46.065823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:46.066472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:46.066758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:46.067032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:46.067677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:46.069852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:46.080502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 977/1952] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:46.085453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:46.095552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65669025-13ab-4ae5-8625-46c905214f47 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:46.100344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:46.101211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:46.101495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:46.101777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] PATH_INFO: `/v3/projects/8e2faf25024746ae8d7d7dccf90c742f/users/46317d823a9b4a878896973b65b91658/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:46.116909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2faf25024746ae8d7d7dccf90c742f, user_id=46317d823a9b4a878896973b65b91658, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:46.119810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:46.195462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 976/1953] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:46 2022] PUT /identity/v3/projects/8e2faf25024746ae8d7d7dccf90c742f/users/46317d823a9b4a878896973b65b91658/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:46.199502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:46.210284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a94b19-1094-46fd-a79c-293dbc795102 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:46.264753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e475a-501a-4435-8f17-fdddd0455992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:46.265358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e475a-501a-4435-8f17-fdddd0455992 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:46.265641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e475a-501a-4435-8f17-fdddd0455992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:46.265913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e475a-501a-4435-8f17-fdddd0455992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:46.266581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e475a-501a-4435-8f17-fdddd0455992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:46.268183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e475a-501a-4435-8f17-fdddd0455992 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:46.278386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 978/1954] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:46.283003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:46.293326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b90ab-2b8d-4ea0-aa73-4cc6314be3fb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:46.298045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:46.298793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:46.299066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:46.299334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] PATH_INFO: `/v3/projects/8e2faf25024746ae8d7d7dccf90c742f/users/46317d823a9b4a878896973b65b91658/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:46.301284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2faf25024746ae8d7d7dccf90c742f, user_id=46317d823a9b4a878896973b65b91658, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:46.304393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:46.314971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 977/1955] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:46 2022] PUT /identity/v3/projects/8e2faf25024746ae8d7d7dccf90c742f/users/46317d823a9b4a878896973b65b91658/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:46.325707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:46.325995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:46.326271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:46.354439 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] MFA Rules not processed for user `46317d823a9b4a878896973b65b91658`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:46.355663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:46.382378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:46.425992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 979/1956] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:46.440382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:46.442081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:46.502912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de9723-6f86-4018-97e9-2a599f714d14 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:46.507039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:46.507649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:46.507924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:46.508187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:46.565576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:46.567206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:46.582542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 978/1957] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:55.064469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:55.075068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062eded-cbf5-46d1-89e0-dfdb6d3a2562 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:55.078694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02cd00-e725-4aba-801f-16f454be33c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:55.079293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02cd00-e725-4aba-801f-16f454be33c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:55.079569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02cd00-e725-4aba-801f-16f454be33c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:55.079839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02cd00-e725-4aba-801f-16f454be33c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:55.089187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02cd00-e725-4aba-801f-16f454be33c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:55.090873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02cd00-e725-4aba-801f-16f454be33c8 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:55.101265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 980/1958] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:55.671367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:55.672950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:55.689746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:55.776283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a6679-4b73-4a38-8616-abf0bc567c4c service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:55.782139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:55.783336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:55.785518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:55.785518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] PATH_INFO: `/v3/users/f50f82989dab48158c16e92f1b1efa7a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:55.785518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50f82989dab48158c16e92f1b1efa7a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:55.787363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:55.957882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50f82989dab48158c16e92f1b1efa7a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:55.958564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50f82989dab48158c16e92f1b1efa7a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:55.971395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50f82989dab48158c16e92f1b1efa7a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:55.978522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50f82989dab48158c16e92f1b1efa7a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:56.091767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50f82989dab48158c16e92f1b1efa7a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:56.098219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 979/1959] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:34:55 2022] DELETE /identity/v3/users/f50f82989dab48158c16e92f1b1efa7a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:56.102941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0a51-c271-423d-9bed-e3f14ca42c50 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:56.103553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0a51-c271-423d-9bed-e3f14ca42c50 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:56.103553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e0a51-c271-423d-9bed-e3f14ca42c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:56.122629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1d7e0a51-c271-423d-9bed-e3f14ca42c50 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:56.123859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1d7e0a51-c271-423d-9bed-e3f14ca42c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:56.189980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 981/1960] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:56.201965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:56.203915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:56.291029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c06ea-6990-4b33-8025-1e65f7da4321 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:56.295145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:56.295791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:56.296070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:56.296475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:56.309304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:56.311069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:56.322865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 980/1961] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:56.740400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618814d8-df37-40d9-b667-c94715267223 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:56.740936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618814d8-df37-40d9-b667-c94715267223 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:56.741185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618814d8-df37-40d9-b667-c94715267223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:56.782410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-618814d8-df37-40d9-b667-c94715267223 None None] MFA Rules not processed for user `205b8f60af7a4957a578f21f9ea464dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:56.782410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-618814d8-df37-40d9-b667-c94715267223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:56.890751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 982/1962] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:56.974463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:56.977642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:57.000739 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ed125-ea98-4d4f-a098-9fa66cf47f3d service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.006690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.007559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.007859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.008065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] PATH_INFO: `/v3/projects/7ef3228c9b934185bce8b64f3415e6fd` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.010498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef3228c9b934185bce8b64f3415e6fd)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.012423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:57.119850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:57.138584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef3228c9b934185bce8b64f3415e6fd', 'request_id': 'req-f143ab39-c442-4830-9272-25d5c28e565c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:57.228402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef3228c9b934185bce8b64f3415e6fd', 'request_id': 'req-f143ab39-c442-4830-9272-25d5c28e565c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:57.239606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef3228c9b934185bce8b64f3415e6fd', 'request_id': 'req-f143ab39-c442-4830-9272-25d5c28e565c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:57.252852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] The token cache is being invalidate because project 7ef3228c9b934185bce8b64f3415e6fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:34:57.253160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:57.253821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f143ab39-c442-4830-9272-25d5c28e565c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:34:57.260567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 981/1963] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:34:56 2022] DELETE /identity/v3/projects/7ef3228c9b934185bce8b64f3415e6fd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:57.264541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.265548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.266098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.288958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:57.290226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:57.345035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:57.358359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 983/1964] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:57.366885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.367196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.367471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.390241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:57.391429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:57.402383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 982/1965] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:57.404011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:57.415019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a98b3-0981-4e0f-8c4f-b42a41109d1b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.420875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.421589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.422091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.422499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.423244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:34:57.423646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.425339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:57.454748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 984/1966] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:34:57.459676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:57.470497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f806d90c-eb20-48ac-a962-1353aad80c4e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.478879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.479506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.479788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.480063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.480722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.483491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:57.620555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 983/1967] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:57 2022] POST /identity/v3/projects => generated 403 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:34:57.624393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:57.635189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f61c3-5a32-48a0-907d-e7e9de73ce88 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.713066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.714177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.714177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.714521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.715255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.717300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:57.835800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 985/1968] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:57 2022] POST /identity/v3/users => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:57.839991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:57.851916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d63d22-24a7-4e38-aaea-4f8e703f75eb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.858331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.859011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.859295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.859566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.860218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.861886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:57.873421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 984/1969] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:34:57.880262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:57.890927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad2d69-1ffc-45fb-900b-e00b5e77d431 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.895621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6064b-2811-4eef-998b-951d74535478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.896225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6064b-2811-4eef-998b-951d74535478 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.897084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6064b-2811-4eef-998b-951d74535478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.897358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6064b-2811-4eef-998b-951d74535478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.898005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6064b-2811-4eef-998b-951d74535478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.899674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6064b-2811-4eef-998b-951d74535478 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:57.916312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 986/1970] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:34:57.919789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:57.932640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4ea8-ebd0-49c0-a874-bf5f5bbc18fa admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:57.937289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:57.937945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:57.938229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:57.938539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] PATH_INFO: `/v3/projects/81a9fa76cb024bafbfd27e0d260b2720/users/585a3a00220d4b14b4b3998cba20f05e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:57.955801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a9fa76cb024bafbfd27e0d260b2720, user_id=585a3a00220d4b14b4b3998cba20f05e, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:57.958554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:58.065868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 985/1971] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:34:57 2022] PUT /identity/v3/projects/81a9fa76cb024bafbfd27e0d260b2720/users/585a3a00220d4b14b4b3998cba20f05e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:34:58.072675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50596-e94a-45b3-a58d-bfe4eee2fbf7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:58.073202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50596-e94a-45b3-a58d-bfe4eee2fbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:58.073753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d50596-e94a-45b3-a58d-bfe4eee2fbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:58.096893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f3d50596-e94a-45b3-a58d-bfe4eee2fbf7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:58.098111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f3d50596-e94a-45b3-a58d-bfe4eee2fbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:58.173609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 987/1972] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:34:58.184091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:58.185740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:58.249411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7156791-3da6-46e3-b278-35d9969612c9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:58.254223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:58.255107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:58.255506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:58.256701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:58.266772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:58.268396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:58.279642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 986/1973] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:34:59.641497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:59.642020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:59.642110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:59.673809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] MFA Rules not processed for user `585a3a00220d4b14b4b3998cba20f05e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:34:59.675337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:34:59.725154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:34:59.742814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 988/1974] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:34:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:34:59.747156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:59.757619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f41646-a0c1-4b12-8018-98529b4ed01d service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:59.762474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:59.763079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:59.763364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:59.763640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:59.764289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:59.766206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:34:59.848827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 987/1975] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:34:59 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:34:59.853247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:34:59.867486 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d8a45-d844-42a5-add2-7d00c78d6350 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:34:59.942135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:34:59.942597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:34:59.942882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:34:59.943741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:34:59.944453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:34:59.946654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:00.048001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 989/1976] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:34:59 2022] POST /identity/v3/users => generated 308 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:35:00.053517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:00.064660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50624df5-e408-438d-b5eb-dfc5ef94f542 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:00.070761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:00.071527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:00.071909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:00.072442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:00.073046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:00.075886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:00.088804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 988/1977] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:00.094427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:00.107447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fd112-2ad8-4759-b543-a956a45bbdd3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:00.112756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:00.113789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:00.114217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:00.114614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] PATH_INFO: `/v3/projects/a68ddcb6b3b4480aa3636b71369e2eaf/users/3dd8687d05e14d7f8fb065d99d61bfe8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:00.137940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68ddcb6b3b4480aa3636b71369e2eaf, user_id=3dd8687d05e14d7f8fb065d99d61bfe8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:00.140699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:00.264103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 990/1978] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:00 2022] PUT /identity/v3/projects/a68ddcb6b3b4480aa3636b71369e2eaf/users/3dd8687d05e14d7f8fb065d99d61bfe8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:00.268498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:00.283679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ad38d-f42e-4da3-b970-64de874af166 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:00.308706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:00.346697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549362ef-b049-4e51-b43b-e99468dabd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:00.347898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549362ef-b049-4e51-b43b-e99468dabd42 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:00.348291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549362ef-b049-4e51-b43b-e99468dabd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:00.348723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549362ef-b049-4e51-b43b-e99468dabd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:00.350300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549362ef-b049-4e51-b43b-e99468dabd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:00.352751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549362ef-b049-4e51-b43b-e99468dabd42 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:00.364296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 989/1979] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:00.369511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:00.380136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e96952-55d9-46ef-b3dd-a038a41cc905 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:00.386524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68088b2c-ec2d-400c-acda-d248a3816135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:00.387282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68088b2c-ec2d-400c-acda-d248a3816135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:00.388125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68088b2c-ec2d-400c-acda-d248a3816135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:00.388406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68088b2c-ec2d-400c-acda-d248a3816135 admin admin] PATH_INFO: `/v3/projects/a68ddcb6b3b4480aa3636b71369e2eaf/users/3dd8687d05e14d7f8fb065d99d61bfe8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:00.392288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68088b2c-ec2d-400c-acda-d248a3816135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a68ddcb6b3b4480aa3636b71369e2eaf, user_id=3dd8687d05e14d7f8fb065d99d61bfe8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:00.399368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68088b2c-ec2d-400c-acda-d248a3816135 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:00.414419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 991/1980] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:00 2022] PUT /identity/v3/projects/a68ddcb6b3b4480aa3636b71369e2eaf/users/3dd8687d05e14d7f8fb065d99d61bfe8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:01.663367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80456956-ca93-450a-9276-53809ac4553f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:01.663875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80456956-ca93-450a-9276-53809ac4553f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:01.664036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80456956-ca93-450a-9276-53809ac4553f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:01.697560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-80456956-ca93-450a-9276-53809ac4553f None None] MFA Rules not processed for user `3dd8687d05e14d7f8fb065d99d61bfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:01.703352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-80456956-ca93-450a-9276-53809ac4553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:01.782502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 990/1981] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:02.465326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0fecb-6b98-4947-916f-f49f32d83fbc None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:02.466719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0fecb-6b98-4947-916f-f49f32d83fbc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:02.466719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0fecb-6b98-4947-916f-f49f32d83fbc None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:02.467534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 992/1982] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:35:02.497415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:02.498054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:02.498183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:02.518286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:02.519637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:02.588034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 991/1983] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:35:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:02.602713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0fecb-6b98-4947-916f-f49f32d83fbc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:02.607900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c7e0fecb-6b98-4947-916f-f49f32d83fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:02.627594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0fecb-6b98-4947-916f-f49f32d83fbc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:02.632899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:02.633510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:02.633790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:02.634060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:02.643643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:02.648781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:02.660551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 993/1984] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:03.277255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:03.279210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:03.354493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:03.383543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595060c4-e2b4-4de2-8152-9efc01a4dbd7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:03.393538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:03.394418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:03.394745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:03.395019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] PATH_INFO: `/v3/users/fd74c9e834d2404d9ea054f2619f1ee9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:03.396986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd74c9e834d2404d9ea054f2619f1ee9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:03.399117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:03.417897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:03.435302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82814b-1972-40f4-94d2-dcbd435cfa4f service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:03.440480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:03.447308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:03.447308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:03.447308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] PATH_INFO: `/v3/users/585a3a00220d4b14b4b3998cba20f05e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:03.447308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] RBAC: Authorizing `identity:delete_user(user_id=585a3a00220d4b14b4b3998cba20f05e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:03.447308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:03.588725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd74c9e834d2404d9ea054f2619f1ee9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.589391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd74c9e834d2404d9ea054f2619f1ee9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.602129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd74c9e834d2404d9ea054f2619f1ee9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.609305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd74c9e834d2404d9ea054f2619f1ee9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.665518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '585a3a00220d4b14b4b3998cba20f05e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.686287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-92028267-85b0-4f85-af52-19133b52124a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd74c9e834d2404d9ea054f2619f1ee9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.698100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 992/1985] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:03 2022] DELETE /identity/v3/users/fd74c9e834d2404d9ea054f2619f1ee9 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:03.703228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:03.703532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:03.703801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:03.742060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:03.743877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:03.784135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '585a3a00220d4b14b4b3998cba20f05e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.786560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '585a3a00220d4b14b4b3998cba20f05e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.828721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '585a3a00220d4b14b4b3998cba20f05e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.840579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '585a3a00220d4b14b4b3998cba20f05e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:03.848039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 994/1986] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:03 2022] DELETE /identity/v3/users/585a3a00220d4b14b4b3998cba20f05e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:03.869653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 993/1987] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:03.879713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:03.882105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:03.887817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:03.964319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41468b8c-f11b-4837-9533-0833aaec973b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:03.968896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:03.969520 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:03.969803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:03.970070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:04.048093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:04.049552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:04.059804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 995/1988] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:04.149995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:04.151679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:04.168576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d06cd-a44e-4edc-b337-e54cee315aaf None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:04.175990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:04.176743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:04.177205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:04.177934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] PATH_INFO: `/v3/projects/81a9fa76cb024bafbfd27e0d260b2720` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:04.179509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a9fa76cb024bafbfd27e0d260b2720)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:04.182048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:04.312608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a9fa76cb024bafbfd27e0d260b2720', 'request_id': 'req-ad305b92-eb0e-478e-b483-98e9cb41661b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.398847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a9fa76cb024bafbfd27e0d260b2720', 'request_id': 'req-ad305b92-eb0e-478e-b483-98e9cb41661b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.402513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:04.412850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a9fa76cb024bafbfd27e0d260b2720', 'request_id': 'req-ad305b92-eb0e-478e-b483-98e9cb41661b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.425701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] The token cache is being invalidate because project 81a9fa76cb024bafbfd27e0d260b2720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:04.426000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.426689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.428553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 994/1989] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:04 2022] DELETE /identity/v3/projects/81a9fa76cb024bafbfd27e0d260b2720 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:04.436917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:04.447912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b04c86a-807e-45bf-a9fe-89b6383cfec8 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:04.488520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:04.490157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:04.522606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:04.523442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:04.523807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:04.524149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] PATH_INFO: `/v3/users/3dd8687d05e14d7f8fb065d99d61bfe8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:04.525775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd8687d05e14d7f8fb065d99d61bfe8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:04.529171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:04.592542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad305b92-eb0e-478e-b483-98e9cb41661b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:04.597379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:04.598035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:04.598326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:04.598627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] PATH_INFO: `/v3/projects/3fe77c11f92542898c50ba656dfc15ba` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:04.600317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe77c11f92542898c50ba656dfc15ba)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:04.602507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:04.772712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd8687d05e14d7f8fb065d99d61bfe8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.773427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe77c11f92542898c50ba656dfc15ba', 'request_id': 'req-90a4d34b-9bc6-4586-8f62-c35c4b650184'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.845568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd8687d05e14d7f8fb065d99d61bfe8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.846191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd8687d05e14d7f8fb065d99d61bfe8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.847686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:04.861650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd8687d05e14d7f8fb065d99d61bfe8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.871391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd8687d05e14d7f8fb065d99d61bfe8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.887369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 996/1990] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:04 2022] DELETE /identity/v3/users/3dd8687d05e14d7f8fb065d99d61bfe8 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:04.983229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe77c11f92542898c50ba656dfc15ba', 'request_id': 'req-90a4d34b-9bc6-4586-8f62-c35c4b650184'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:04.994881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe77c11f92542898c50ba656dfc15ba', 'request_id': 'req-90a4d34b-9bc6-4586-8f62-c35c4b650184'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.010207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] The token cache is being invalidate because project 3fe77c11f92542898c50ba656dfc15ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:05.010904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.011771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.016808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 995/1991] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:04 2022] DELETE /identity/v3/projects/3fe77c11f92542898c50ba656dfc15ba => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:05.022876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:05.022876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:05.097274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af18578-483d-454f-a3a8-3d44a4715117 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:05.103502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:05.104174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:05.104463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:05.104729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] PATH_INFO: `/v3/users/46317d823a9b4a878896973b65b91658` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:05.106001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46317d823a9b4a878896973b65b91658)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:05.108453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:05.291925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46317d823a9b4a878896973b65b91658'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.366608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46317d823a9b4a878896973b65b91658'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.367966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46317d823a9b4a878896973b65b91658'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.381405 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:05.384643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:05.392013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46317d823a9b4a878896973b65b91658'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.414869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1a422e5-ffd6-476c-82f3-1630e77f277e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46317d823a9b4a878896973b65b91658'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.422244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 997/1992] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:05 2022] DELETE /identity/v3/users/46317d823a9b4a878896973b65b91658 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:05.512226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4d34b-9bc6-4586-8f62-c35c4b650184 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:05.595106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:05.597373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:05.597874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:05.598130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] PATH_INFO: `/v3/projects/a68ddcb6b3b4480aa3636b71369e2eaf` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:05.599559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a68ddcb6b3b4480aa3636b71369e2eaf)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:05.602450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:05.774289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a68ddcb6b3b4480aa3636b71369e2eaf', 'request_id': 'req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.855985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a68ddcb6b3b4480aa3636b71369e2eaf', 'request_id': 'req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.870609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a68ddcb6b3b4480aa3636b71369e2eaf', 'request_id': 'req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.882185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:05.905354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] The token cache is being invalidate because project a68ddcb6b3b4480aa3636b71369e2eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:05.905354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.907627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c7d7386e-7da2-4a84-b7a1-386253b0a8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:05.910045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 996/1993] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:05 2022] DELETE /identity/v3/projects/a68ddcb6b3b4480aa3636b71369e2eaf => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:05.917509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:05.918018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:05.918481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:05.945802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:05.947371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.020460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 998/1994] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:06.028472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.033039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.033039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.051983 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:06.056077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.063668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 997/1995] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:06.067991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.081321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d453a0-32e8-490d-bfd8-dcf2fa27c6ab None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.087080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.087708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.087991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.088265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.088928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:06.089234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.090975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.113521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 999/1996] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:35:06.113986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.121370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc428c6d-b01f-4396-afe4-45fcf7c4dcd7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.124433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.125073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.126074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.126074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.126546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.128210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.244916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.246732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.260970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c15606-1429-44c5-933b-050f322abe00 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.267624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.268609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.269150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.269596 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] PATH_INFO: `/v3/projects/8e2faf25024746ae8d7d7dccf90c742f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.271013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2faf25024746ae8d7d7dccf90c742f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.279842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 998/1997] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:06.280319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.284610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.388540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259b0fb-3d88-4f9e-ae8f-1df2a7e4689b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.408819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2faf25024746ae8d7d7dccf90c742f', 'request_id': 'req-5f1b42a5-c003-428e-a3b5-dff6b910b50c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:06.454491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.455104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.455386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.455684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.456387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.458401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.507983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2faf25024746ae8d7d7dccf90c742f', 'request_id': 'req-5f1b42a5-c003-428e-a3b5-dff6b910b50c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:06.514022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2faf25024746ae8d7d7dccf90c742f', 'request_id': 'req-5f1b42a5-c003-428e-a3b5-dff6b910b50c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:06.526488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] The token cache is being invalidate because project 8e2faf25024746ae8d7d7dccf90c742f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:06.526775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:06.527436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5f1b42a5-c003-428e-a3b5-dff6b910b50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:06.529239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1000/1998] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:06 2022] DELETE /identity/v3/projects/8e2faf25024746ae8d7d7dccf90c742f => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:06.534880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95e58d-8c07-4778-8217-56f0e90e7536 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.535171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95e58d-8c07-4778-8217-56f0e90e7536 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.535451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95e58d-8c07-4778-8217-56f0e90e7536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.556187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ae95e58d-8c07-4778-8217-56f0e90e7536 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:06.557688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ae95e58d-8c07-4778-8217-56f0e90e7536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.567301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1001/1999] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:06.580892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.581313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.581670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.586194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 999/2000] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/users => generated 284 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:06.590591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.594651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.607523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:06.610782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.618611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c0408-8776-4d0c-b951-baf003a8621c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.627170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.628751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1002/2001] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:06.629058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.630140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.630408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.630963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.632861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.635107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.644294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e0f29-8ec5-45be-8d66-0858cf0efd9b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.649124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.649744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.650026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.650426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.651059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:06.651358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.653040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.654385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1000/2002] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:06.659387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.668851 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1003/2003] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:35:06.672833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6d66b-ce1d-4c8c-8174-0f7a89bbe410 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.676650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.679415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.680262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.680690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.680972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.681771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.683482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.689024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811dbf0-87f3-4cb0-b869-c957e523f732 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.695443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1574e-3689-4774-b665-7c5488bac1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.696254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1574e-3689-4774-b665-7c5488bac1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.696605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1574e-3689-4774-b665-7c5488bac1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.696948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1574e-3689-4774-b665-7c5488bac1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.697741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1574e-3689-4774-b665-7c5488bac1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.699290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1001/2004] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:06.699847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1574e-3689-4774-b665-7c5488bac1d1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.703312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.716566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf78cf0-2d56-40cc-9938-7f5fce29e4ed admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.722867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.725090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.725090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.725548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] PATH_INFO: `/v3/projects/0cf4fa11664043108646f2beae269433/users/ee71afee031a4e598ffdb84a08148302/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.742710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf4fa11664043108646f2beae269433, user_id=ee71afee031a4e598ffdb84a08148302, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.745367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:06.794104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:06.830791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1004/2005] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:35:06.830791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb81e4-47db-474c-bfae-164267a6dd55 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.830791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb81e4-47db-474c-bfae-164267a6dd55 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.830791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb81e4-47db-474c-bfae-164267a6dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.835814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1002/2006] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:06 2022] PUT /identity/v3/projects/0cf4fa11664043108646f2beae269433/users/ee71afee031a4e598ffdb84a08148302/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:06.836816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:06.857362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0ddb81e4-47db-474c-bfae-164267a6dd55 None None] MFA Rules not processed for user `409c324dca974d9dad9529b2a0d82b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:06.858886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0ddb81e4-47db-474c-bfae-164267a6dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.914079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52fd08-d7ac-4474-92a8-4269095c7063 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:06.916952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1005/2007] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:06.918204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.918760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.918760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.952179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] MFA Rules not processed for user `ee71afee031a4e598ffdb84a08148302`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:06.953563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:06.985648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:06.986278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:06.986590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:06.986861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:06.987493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:06.989553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.044963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1006/2008] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:07.049609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.061161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743add5-4878-4e09-b91a-c864a2126e1d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.065876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.066521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.066802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.067069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.067704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.069770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.108931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1003/2009] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:06 2022] POST /identity/v3/users => generated 288 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:07.113380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.144073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4ba5b-8e10-425c-9bf3-d5201e7cdffd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.153085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.153719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.154006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.154285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.154982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.156866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.183797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1004/2010] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:07.186096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.199479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866855dd-e2bb-4dac-811e-70ad3387c99e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.204663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1007/2011] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:07 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:35:07.205982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.207131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.207993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.208474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.209303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.210181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.212267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.225721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1005/2012] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:07.229363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.239276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedcb21-1dfd-4a10-bdaf-3c33031e1934 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.292588 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5503678-6cee-48c6-ac7b-7c524462d2c4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.299939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.300859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.301526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.301816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.302849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.304916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.316748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef11aa-3d0b-4e5b-bf96-dafca515efc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.318558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef11aa-3d0b-4e5b-bf96-dafca515efc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.318954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef11aa-3d0b-4e5b-bf96-dafca515efc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.319334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef11aa-3d0b-4e5b-bf96-dafca515efc2 admin admin] PATH_INFO: `/v3/projects/f3e78f7c51b14f348b670dde8732c61e/users/2881400b39a147a8a92d873cf6ceae89/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.337545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef11aa-3d0b-4e5b-bf96-dafca515efc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e78f7c51b14f348b670dde8732c61e, user_id=2881400b39a147a8a92d873cf6ceae89, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.340043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef11aa-3d0b-4e5b-bf96-dafca515efc2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.467280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1008/2013] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:07 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:07.468767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.479982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc119f-6307-49ad-8aff-3b86522c1a86 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.484872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.485894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.485894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.486133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.486821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.488432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.499233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1009/2014] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:07.503172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.513680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a9736-9b41-42a2-95f8-69ad1da5ca58 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.518622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.519358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.519765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.520103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] PATH_INFO: `/v3/projects/c28161f1430e4cab9261a454f8ef748e/users/3590377e80194bad91e9b299ea14cd3e/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.536148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28161f1430e4cab9261a454f8ef748e, user_id=3590377e80194bad91e9b299ea14cd3e, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.542394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.835523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1006/2015] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:07 2022] PUT /identity/v3/projects/f3e78f7c51b14f348b670dde8732c61e/users/2881400b39a147a8a92d873cf6ceae89/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:07.846696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.847002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.847270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.861952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1010/2016] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:07 2022] PUT /identity/v3/projects/c28161f1430e4cab9261a454f8ef748e/users/3590377e80194bad91e9b299ea14cd3e/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:07.865788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:07.899374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] MFA Rules not processed for user `2881400b39a147a8a92d873cf6ceae89`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:07.900166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:07.942481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb26ae-967f-441d-95f3-108702eb2a31 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:07.949716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:07.951908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:07.952218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:07.952544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:07.953204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:07.955425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:07.984285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1011/2017] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:07.990839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:08.004239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42bef2-96e3-46c0-be72-ba85e0996b2c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:08.004371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:08.007370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1007/2018] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:08.021608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:08.023120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af375e6c-834c-42f3-a230-95844fb5b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:08.024205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af375e6c-834c-42f3-a230-95844fb5b7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:08.024655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af375e6c-834c-42f3-a230-95844fb5b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:08.024780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:08.026885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af375e6c-834c-42f3-a230-95844fb5b7ae admin admin] PATH_INFO: `/v3/projects/c28161f1430e4cab9261a454f8ef748e/users/3590377e80194bad91e9b299ea14cd3e/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:08.032144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af375e6c-834c-42f3-a230-95844fb5b7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28161f1430e4cab9261a454f8ef748e, user_id=3590377e80194bad91e9b299ea14cd3e, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:08.036559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af375e6c-834c-42f3-a230-95844fb5b7ae admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:08.052024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1012/2019] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:07 2022] PUT /identity/v3/projects/c28161f1430e4cab9261a454f8ef748e/users/3590377e80194bad91e9b299ea14cd3e/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:08.064022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:08.065055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:08.065443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:08.105036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480dbdb-be57-4e12-a36e-d30eb6763b87 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:08.112112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] MFA Rules not processed for user `3590377e80194bad91e9b299ea14cd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:08.113444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:08.190302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:08.191228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:08.191582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:08.191925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:08.211100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1013/2020] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:08.219260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:08.221850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:08.248264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbc68f-8b77-45c0-b8c9-3f5e91e3a789 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:08.263590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:08.264044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:08.264433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:08.266273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:08.281364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:08.283487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:08.286186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:08.294696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1014/2021] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:35:08.297837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:08.313182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1008/2022] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:08.451646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:08.464741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be576315-5789-4ee8-b5db-60ecd7f0b5c3 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:08.474857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b485fb-5b66-489a-8fee-67d5d8e4c1a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:08.476319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b485fb-5b66-489a-8fee-67d5d8e4c1a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:08.476951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b485fb-5b66-489a-8fee-67d5d8e4c1a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:08.477718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b485fb-5b66-489a-8fee-67d5d8e4c1a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:08.546687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b485fb-5b66-489a-8fee-67d5d8e4c1a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:08.548474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b485fb-5b66-489a-8fee-67d5d8e4c1a9 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:08.566466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1015/2023] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:08.864142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:09.136726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:09.138210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:09.254382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb247be-d625-4780-a18f-5efc89464b9c service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:09.259933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:09.260690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:09.261026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:09.261637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] PATH_INFO: `/v3/users/0f4e4d46d46847c1af305dc028a8da6f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:09.262835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4e4d46d46847c1af305dc028a8da6f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:09.265803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:09.480830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4e4d46d46847c1af305dc028a8da6f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:09.481552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4e4d46d46847c1af305dc028a8da6f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:09.494668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4e4d46d46847c1af305dc028a8da6f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:09.502104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4e4d46d46847c1af305dc028a8da6f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:09.576878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4e4d46d46847c1af305dc028a8da6f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:09.585344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1009/2024] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:09 2022] DELETE /identity/v3/users/0f4e4d46d46847c1af305dc028a8da6f => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:09.590022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:09.590587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:09.590587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:09.614199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:09.615973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:09.687856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1016/2025] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:09.696708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:09.698208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:09.765242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f9e8a-ae82-497d-9401-6de0f20f912f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:09.770158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bfdae-b556-4537-bb58-8dcee969b45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:09.770962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bfdae-b556-4537-bb58-8dcee969b45e service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:09.771893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bfdae-b556-4537-bb58-8dcee969b45e service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:09.772201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bfdae-b556-4537-bb58-8dcee969b45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:09.783027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bfdae-b556-4537-bb58-8dcee969b45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:09.784893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bfdae-b556-4537-bb58-8dcee969b45e service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:09.796527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1010/2026] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:10.159790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:10.273886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:10.275594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:10.328199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133df435-c36c-4f9a-b4c5-76860530a886 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:10.335015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:10.335861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:10.336213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:10.336553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] PATH_INFO: `/v3/projects/ae01041111db4aefb2647f5b062d5ff3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:10.337927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae01041111db4aefb2647f5b062d5ff3)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:10.340925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:10.527981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae01041111db4aefb2647f5b062d5ff3', 'request_id': 'req-01cf0fb2-23de-429a-8330-ee6da57d91dc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:10.618761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae01041111db4aefb2647f5b062d5ff3', 'request_id': 'req-01cf0fb2-23de-429a-8330-ee6da57d91dc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:10.627081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae01041111db4aefb2647f5b062d5ff3', 'request_id': 'req-01cf0fb2-23de-429a-8330-ee6da57d91dc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:10.658872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] The token cache is being invalidate because project ae01041111db4aefb2647f5b062d5ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:10.659167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:10.659825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-01cf0fb2-23de-429a-8330-ee6da57d91dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:10.662640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1017/2027] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:10 2022] DELETE /identity/v3/projects/ae01041111db4aefb2647f5b062d5ff3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:10.667427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:10.668035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:10.668316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:10.689346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:10.690593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:10.759501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1011/2028] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:10.775823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:10.775823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:10.775823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:10.807429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:10.807429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:10.818845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1018/2029] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:10.823655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:10.836100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6970d3-9a58-4fd3-91ba-9620a98ce470 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:10.842626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:10.843808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:10.844316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:10.844689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:10.845503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:10.845902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:10.847765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:10.866676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1012/2030] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:35:10.875005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:10.886994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6adb6-b7de-4333-b31a-e2ae43690785 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:10.898427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:10.899988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:10.900721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:10.901505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:10.903193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:10.925563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:11.060141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:11.060646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1019/2031] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:10 2022] POST /identity/v3/projects => generated 369 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:11.072821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4bf20-e9ab-41d0-aabe-c6c70f9eaf48 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:11.147569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:11.148164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:11.148441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:11.148710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:11.149379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:11.151470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:11.325024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1013/2032] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:11 2022] POST /identity/v3/users => generated 285 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:11.327456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:11.339445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ebb-0f77-4662-8231-f3c949e07ab8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:11.345841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:11.346516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:11.346797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:11.347075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:11.347836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:11.349951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:11.364778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1020/2033] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:11.366523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:11.379566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb62d81-c2e2-4a76-89f5-f78af77f3560 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:11.386581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a677e-0828-439e-85ac-8fb847a7f649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:11.387323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a677e-0828-439e-85ac-8fb847a7f649 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:11.387713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a677e-0828-439e-85ac-8fb847a7f649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:11.388091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a677e-0828-439e-85ac-8fb847a7f649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:11.388855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a677e-0828-439e-85ac-8fb847a7f649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:11.390698 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a677e-0828-439e-85ac-8fb847a7f649 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:11.413156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1014/2034] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:11.416963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:11.426693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:11.435557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7aea5-f76a-4e8f-9b53-4a3a7b74efd7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:11.443714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:11.444623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:11.445011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:11.445545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] PATH_INFO: `/v3/projects/d9ad6a82910044ca9d6a287d687cac8b/users/5c3a7ab81e3a4c3dbc7d7234fbb3ba5a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:11.463583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ad6a82910044ca9d6a287d687cac8b, user_id=5c3a7ab81e3a4c3dbc7d7234fbb3ba5a, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:11.466615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:11.538763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1021/2035] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:11 2022] PUT /identity/v3/projects/d9ad6a82910044ca9d6a287d687cac8b/users/5c3a7ab81e3a4c3dbc7d7234fbb3ba5a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:11.550092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:11.550482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:11.550884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:11.563213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:11.607266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] MFA Rules not processed for user `5c3a7ab81e3a4c3dbc7d7234fbb3ba5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:11.608870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:11.687970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1015/2036] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:11.702293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:11.704356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:11.770580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987202e-d7e5-4436-817d-7b113690418d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:11.775904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d84c67-91ea-45b4-ba8b-bc53b4dd12ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:11.776732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d84c67-91ea-45b4-ba8b-bc53b4dd12ea service glance] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:11.777334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d84c67-91ea-45b4-ba8b-bc53b4dd12ea service glance] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:11.777803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d84c67-91ea-45b4-ba8b-bc53b4dd12ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:11.789916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d84c67-91ea-45b4-ba8b-bc53b4dd12ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:11.791716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d84c67-91ea-45b4-ba8b-bc53b4dd12ea service glance] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:11.806448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1022/2037] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:12.165973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:12.372482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:13.481043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:13.840710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:14.751732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:14.753194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:14.850045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3093f-ce87-4a67-9767-5fffbe76af54 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:14.865846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:14.867244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:14.867738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:14.868170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] PATH_INFO: `/v3/users/ee71afee031a4e598ffdb84a08148302` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:14.869741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee71afee031a4e598ffdb84a08148302)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:14.872513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:15.065986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee71afee031a4e598ffdb84a08148302'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:15.066753 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee71afee031a4e598ffdb84a08148302'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:15.080709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee71afee031a4e598ffdb84a08148302'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:15.088487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee71afee031a4e598ffdb84a08148302'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:15.149860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee71afee031a4e598ffdb84a08148302'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:15.158051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1016/2038] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:14 2022] DELETE /identity/v3/users/ee71afee031a4e598ffdb84a08148302 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:15.162747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:15.163249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:15.163367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:15.182423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:15.183653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:15.250853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1023/2039] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:15.261870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:15.263390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:15.326249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48115ec3-1334-4c41-9ce3-dfa4e73dacb0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:15.330289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:15.331073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:15.331418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:15.331756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:15.344623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:15.346308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:15.356705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1017/2040] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:15.389090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:15.847551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:15.849277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:15.862229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faeb460-f21f-4fae-9426-e0a4072ab5c0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:15.867275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:15.867946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:15.869085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:15.869368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] PATH_INFO: `/v3/projects/0cf4fa11664043108646f2beae269433` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:15.870599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf4fa11664043108646f2beae269433)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:15.872594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:16.042073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf4fa11664043108646f2beae269433', 'request_id': 'req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.182500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf4fa11664043108646f2beae269433', 'request_id': 'req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.197579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf4fa11664043108646f2beae269433', 'request_id': 'req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.210626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] The token cache is being invalidate because project 0cf4fa11664043108646f2beae269433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:16.210924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.211592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.213112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1024/2041] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:15 2022] DELETE /identity/v3/projects/0cf4fa11664043108646f2beae269433 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:16.217569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:16.232637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acd944-992c-4f3e-a96e-62ffa8140965 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:16.290601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:16.291174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:16.291451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:16.291722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] PATH_INFO: `/v3/users/3590377e80194bad91e9b299ea14cd3e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:16.292882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3590377e80194bad91e9b299ea14cd3e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:16.294861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:16.557662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3590377e80194bad91e9b299ea14cd3e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.558790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3590377e80194bad91e9b299ea14cd3e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.574561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3590377e80194bad91e9b299ea14cd3e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.584262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3590377e80194bad91e9b299ea14cd3e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.674254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f879a496-9183-4499-acce-7817e992d544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3590377e80194bad91e9b299ea14cd3e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:16.684685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1018/2042] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:16 2022] DELETE /identity/v3/users/3590377e80194bad91e9b299ea14cd3e => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:17.277578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:17.287764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:17.310396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78c9b-138d-4f2d-80f9-7a34ea71b2b5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:17.405575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:17.406124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:17.406587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:17.406829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] PATH_INFO: `/v3/projects/c28161f1430e4cab9261a454f8ef748e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:17.408065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28161f1430e4cab9261a454f8ef748e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:17.410057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:17.451432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:17.572113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28161f1430e4cab9261a454f8ef748e', 'request_id': 'req-92c8ba76-467b-4a01-98b3-b8ce929a86b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:17.634227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28161f1430e4cab9261a454f8ef748e', 'request_id': 'req-92c8ba76-467b-4a01-98b3-b8ce929a86b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:17.647119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28161f1430e4cab9261a454f8ef748e', 'request_id': 'req-92c8ba76-467b-4a01-98b3-b8ce929a86b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:17.658186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] The token cache is being invalidate because project c28161f1430e4cab9261a454f8ef748e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:17.658568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:17.659502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92c8ba76-467b-4a01-98b3-b8ce929a86b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:17.661284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1025/2043] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:17 2022] DELETE /identity/v3/projects/c28161f1430e4cab9261a454f8ef748e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:17.667440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:17.667961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:17.668038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:17.691845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:17.691845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:17.718257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:17.751795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1019/2044] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:17.760367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:17.760686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:17.760965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:17.787785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:17.789934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:17.801406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1026/2045] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:17.805079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:17.833811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402cf02-938f-4101-a67e-9aad19796e16 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:17.840090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:17.840850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:17.841342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:17.841614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:17.842274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:17.842612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:17.844216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:17.857685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1020/2046] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:35:17.862451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:17.874719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a2f38-2f98-4051-80dc-2df9d94859c1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:17.879701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:17.881168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:17.881445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:17.881714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:17.882401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:17.885171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:17.995752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1027/2047] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:17 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:35:17.999046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:18.009429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d06281-805a-4387-8cc9-32b16c8a579a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:18.072226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:18.072843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.073124 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.073392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.074060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:18.076056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:18.211876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1021/2048] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:17 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:18.217825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:18.237984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8654a8-f3d8-4cdc-bec9-e45be8a4ffe7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:18.255570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:18.256494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.256853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.257201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.257930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:18.259779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:18.271295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1028/2049] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:18.275468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:18.287067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b5efd-d825-4c5d-82ae-208fb674f87e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:18.295213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a667-957b-4b5d-a22f-652e29b6f416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:18.295213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a667-957b-4b5d-a22f-652e29b6f416 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.295213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a667-957b-4b5d-a22f-652e29b6f416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.295213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a667-957b-4b5d-a22f-652e29b6f416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.295631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a667-957b-4b5d-a22f-652e29b6f416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:18.297480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a667-957b-4b5d-a22f-652e29b6f416 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:18.314383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1022/2050] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:18.317210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:18.329922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee0920-341d-4fc4-869c-10730964f04c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:18.334950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:18.335620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.335900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.336183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] PATH_INFO: `/v3/projects/5e49133c0d5d45969ab8da55590b8c7c/users/0b9414bedf624b73816225742bb244c1/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.356255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e49133c0d5d45969ab8da55590b8c7c, user_id=0b9414bedf624b73816225742bb244c1, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:18.363929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:18.457070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1029/2051] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:18 2022] PUT /identity/v3/projects/5e49133c0d5d45969ab8da55590b8c7c/users/0b9414bedf624b73816225742bb244c1/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:18.468151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.468443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.468718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.499289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] MFA Rules not processed for user `0b9414bedf624b73816225742bb244c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:18.500532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:18.568853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1023/2052] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:18.573502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:18.644139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803302db-78e9-4519-a191-52f65495d5fa admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:18.652771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:18.653397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.653837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.654226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.654776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:18.664149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:18.790209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1030/2053] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:18 2022] POST /identity/v3/projects => generated 391 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:35:18.794183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:18.805547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa471e6b-b899-4767-9697-3b3531f0bd62 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:18.832063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:18.867601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:18.868365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:18.868871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:18.869377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:18.870309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:18.872880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.009788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1024/2054] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:18 2022] POST /identity/v3/users => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:19.012290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:19.026231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9505bb7-b50f-4c0a-9d4d-694b45279962 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:19.031697 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:19.032806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.032909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.033130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.033811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:19.035457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.050047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1031/2055] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:19.054458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:19.064958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d86510-e50a-410d-a540-3bbef20d5b12 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:19.071113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:19.071769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.072218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.072646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] PATH_INFO: `/v3/projects/7a1b472ca74942c1acf8c47b35611f58/users/42a24888234543e38a60b0b4ea4de8e0/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.093378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1b472ca74942c1acf8c47b35611f58, user_id=42a24888234543e38a60b0b4ea4de8e0, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:19.096415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.230025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1025/2056] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:19 2022] PUT /identity/v3/projects/7a1b472ca74942c1acf8c47b35611f58/users/42a24888234543e38a60b0b4ea4de8e0/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:19.236179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:19.250714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e922-9cd5-4dd6-ba75-342ac95471fb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:19.277253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:19.360999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943a6de-622e-40e7-9a80-7127451ecda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:19.361636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a6de-622e-40e7-9a80-7127451ecda7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.361907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a6de-622e-40e7-9a80-7127451ecda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.362205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a6de-622e-40e7-9a80-7127451ecda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.362948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943a6de-622e-40e7-9a80-7127451ecda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:19.369831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943a6de-622e-40e7-9a80-7127451ecda7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.399669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1032/2057] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:19.410429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:19.438410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967b4d8-573b-49e9-a63d-657ada78612c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:19.447034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:19.449140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.450172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.450172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] PATH_INFO: `/v3/projects/7a1b472ca74942c1acf8c47b35611f58/users/42a24888234543e38a60b0b4ea4de8e0/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.455793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1b472ca74942c1acf8c47b35611f58, user_id=42a24888234543e38a60b0b4ea4de8e0, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:19.464853 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.496141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1026/2058] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:19 2022] PUT /identity/v3/projects/7a1b472ca74942c1acf8c47b35611f58/users/42a24888234543e38a60b0b4ea4de8e0/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:19.509514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.510387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.510387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.560515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] MFA Rules not processed for user `42a24888234543e38a60b0b4ea4de8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:19.561755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:19.697875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1033/2059] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:19.712270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:19.714059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:19.776412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278f2df-c379-4db3-8984-36c4741b6579 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:19.780031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:19.780650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.781197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.781477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.792433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:19.793942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.805189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1027/2060] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:19.862742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:19.872850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e11ad-533b-4279-9e6e-618c0858cd3b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:19.876548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd96c92-4eb1-446d-a586-2be9f74a947e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:19.877143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd96c92-4eb1-446d-a586-2be9f74a947e service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:19.877419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd96c92-4eb1-446d-a586-2be9f74a947e service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:19.877688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd96c92-4eb1-446d-a586-2be9f74a947e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:19.939141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd96c92-4eb1-446d-a586-2be9f74a947e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:19.940610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd96c92-4eb1-446d-a586-2be9f74a947e service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:19.953419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1034/2061] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:21.736682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:21.738782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:21.812959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bf032-a833-4ff8-9855-b95362dd5234 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:21.820605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:21.821527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:21.821820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:21.822093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] PATH_INFO: `/v3/users/2881400b39a147a8a92d873cf6ceae89` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:21.823764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2881400b39a147a8a92d873cf6ceae89)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:21.828582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:21.853066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:22.116568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2881400b39a147a8a92d873cf6ceae89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:22.117284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2881400b39a147a8a92d873cf6ceae89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:22.135606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2881400b39a147a8a92d873cf6ceae89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:22.142810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2881400b39a147a8a92d873cf6ceae89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:22.234134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2881400b39a147a8a92d873cf6ceae89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:22.246114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1028/2062] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:21 2022] DELETE /identity/v3/users/2881400b39a147a8a92d873cf6ceae89 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:22.248316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:22.248825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:22.249016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:22.271675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:22.276242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:22.311735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:22.336535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1035/2063] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:22.347154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:22.351644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:22.411681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a6c05-11ce-4130-85ad-b83e43d2cb7b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:22.415309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cfe6b-22e1-410c-9030-ff477cc7018d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:22.415911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cfe6b-22e1-410c-9030-ff477cc7018d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:22.416277 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cfe6b-22e1-410c-9030-ff477cc7018d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:22.416463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cfe6b-22e1-410c-9030-ff477cc7018d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:22.426507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cfe6b-22e1-410c-9030-ff477cc7018d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:22.429542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6cfe6b-22e1-410c-9030-ff477cc7018d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:22.438740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1029/2064] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:22.932323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:22.935549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:22.964502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286a971-3004-4228-a0d1-3a265af75391 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:22.971456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:22.972782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:22.974155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:22.983816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] PATH_INFO: `/v3/projects/f3e78f7c51b14f348b670dde8732c61e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:22.985228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3e78f7c51b14f348b670dde8732c61e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:22.988361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:23.134973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3e78f7c51b14f348b670dde8732c61e', 'request_id': 'req-e7d57a32-6af2-46fe-9e57-1169280f786d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:23.241548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3e78f7c51b14f348b670dde8732c61e', 'request_id': 'req-e7d57a32-6af2-46fe-9e57-1169280f786d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:23.252028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3e78f7c51b14f348b670dde8732c61e', 'request_id': 'req-e7d57a32-6af2-46fe-9e57-1169280f786d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:23.303203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] The token cache is being invalidate because project f3e78f7c51b14f348b670dde8732c61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:23.303758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:23.304722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7d57a32-6af2-46fe-9e57-1169280f786d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:23.307119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1036/2065] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:22 2022] DELETE /identity/v3/projects/f3e78f7c51b14f348b670dde8732c61e => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:23.313554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:23.314220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:23.314830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:23.358057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:23.362481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:23.464649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1030/2066] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:23.466576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:23.466877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:23.467133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:23.499812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:23.501240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:23.511210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1037/2067] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:23.515898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:23.533897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad908f-75a6-4ee1-99d3-c2febc3b7810 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:23.540464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:23.541198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:23.541563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:23.541906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:23.542707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:23.543120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:23.544942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:23.564036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1031/2068] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:35:23.570091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:23.581825 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53251c6-c3b9-4060-9cd1-e2a036b91186 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:23.589364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:23.590531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:23.591026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:23.591451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:23.592264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:23.595710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:23.754862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1038/2069] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:23 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:23.759158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:23.772125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d647c-9aec-4546-8bb4-1af82184e39e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:23.807455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:23.845502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:23.846230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:23.846540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:23.846804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:23.847446 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:23.850264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:23.983731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1032/2070] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:23 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:23.988859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:24.004371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc65a1-a6da-4804-ab48-f7a61530281a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:24.010267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:24.010917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:24.011201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:24.011479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:24.012136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:24.013863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:24.030382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1039/2071] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:24.033211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:24.044790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b597f-f171-44ca-9efc-59096a0db78f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:24.050324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6ce2f-098e-41c3-9a93-7ec50235399b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:24.051123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6ce2f-098e-41c3-9a93-7ec50235399b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:24.051384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6ce2f-098e-41c3-9a93-7ec50235399b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:24.051662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6ce2f-098e-41c3-9a93-7ec50235399b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:24.052539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6ce2f-098e-41c3-9a93-7ec50235399b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:24.054153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6ce2f-098e-41c3-9a93-7ec50235399b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:24.071790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1033/2072] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:24.079159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:24.093020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187caa0-1b50-427c-b8be-00755c7904fc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:24.098801 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:24.099530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:24.099937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:24.100282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] PATH_INFO: `/v3/projects/364812df6abf492698453972e9a1d120/users/b8349bd45ad84f14858f73bf42b48380/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:24.119640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364812df6abf492698453972e9a1d120, user_id=b8349bd45ad84f14858f73bf42b48380, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:24.122752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:24.183052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1040/2073] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:24 2022] PUT /identity/v3/projects/364812df6abf492698453972e9a1d120/users/b8349bd45ad84f14858f73bf42b48380/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:24.188693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:24.189056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:24.189457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:24.210548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:24.211826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:24.273164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1034/2074] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:24.283174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:24.285338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:24.354920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d75f98-bdca-4fb6-8074-5a7388e43a62 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:24.358737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad045a-e1f0-4e71-a692-c1576870c4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:24.360557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad045a-e1f0-4e71-a692-c1576870c4cb service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:24.360557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad045a-e1f0-4e71-a692-c1576870c4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:24.360557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad045a-e1f0-4e71-a692-c1576870c4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:24.370671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad045a-e1f0-4e71-a692-c1576870c4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:24.372227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad045a-e1f0-4e71-a692-c1576870c4cb service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:24.382761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1041/2075] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:25.437569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:25.439608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:25.463150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab4d4-97d0-4004-8560-bc30e6159aab None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:25.475228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:25.476909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:25.477430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:25.477879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] PATH_INFO: `/v3/users/0b9414bedf624b73816225742bb244c1` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:25.480842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9414bedf624b73816225742bb244c1)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:25.498949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:25.690871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9414bedf624b73816225742bb244c1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:25.692023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9414bedf624b73816225742bb244c1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:25.710649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9414bedf624b73816225742bb244c1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:25.718281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9414bedf624b73816225742bb244c1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:25.788485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9414bedf624b73816225742bb244c1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:25.801467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1035/2076] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:25 2022] DELETE /identity/v3/users/0b9414bedf624b73816225742bb244c1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:25.807462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:25.807995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:25.807995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:25.834095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:25.851033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:25.852262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:25.973358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1042/2077] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:25.985547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:25.987772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:26.021369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:26.058274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af62f96-bc45-4f89-99dd-1bde162de271 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:26.062373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:26.063070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:26.063421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:26.063868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:26.080353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:26.081846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:26.095800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1036/2078] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:26.565121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:26.566971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:26.582269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374e1ec-b24f-4bb4-bfcc-89a922bc819b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:26.590256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:26.591073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:26.591668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:26.592131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] PATH_INFO: `/v3/projects/5e49133c0d5d45969ab8da55590b8c7c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:26.594911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e49133c0d5d45969ab8da55590b8c7c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:26.598207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:26.776195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e49133c0d5d45969ab8da55590b8c7c', 'request_id': 'req-9bfbd64d-a94d-4496-8ce1-a5715b58801a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:26.891282 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e49133c0d5d45969ab8da55590b8c7c', 'request_id': 'req-9bfbd64d-a94d-4496-8ce1-a5715b58801a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:26.918138 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e49133c0d5d45969ab8da55590b8c7c', 'request_id': 'req-9bfbd64d-a94d-4496-8ce1-a5715b58801a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:26.965741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] The token cache is being invalidate because project 5e49133c0d5d45969ab8da55590b8c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:26.966467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:26.967354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:26.975292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:26.975684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1043/2079] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:26 2022] DELETE /identity/v3/projects/5e49133c0d5d45969ab8da55590b8c7c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:26.987981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd6762-90dd-4a74-b29e-812024dc118c service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:27.109069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:27.110974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:27.111586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:27.112435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] PATH_INFO: `/v3/users/42a24888234543e38a60b0b4ea4de8e0` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:27.116506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a24888234543e38a60b0b4ea4de8e0)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:27.120378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:27.412452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a24888234543e38a60b0b4ea4de8e0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:27.413143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a24888234543e38a60b0b4ea4de8e0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:27.445872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a24888234543e38a60b0b4ea4de8e0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:27.466616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a24888234543e38a60b0b4ea4de8e0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:27.537934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-84b5d46c-4c13-419f-87eb-1b7039a9a96b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a24888234543e38a60b0b4ea4de8e0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:27.545443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1037/2080] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:26 2022] DELETE /identity/v3/users/42a24888234543e38a60b0b4ea4de8e0 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:28.363762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:28.371013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:28.395164 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbd64d-a94d-4496-8ce1-a5715b58801a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:28.513086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:28.513761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:28.514044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:28.514315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] PATH_INFO: `/v3/projects/7a1b472ca74942c1acf8c47b35611f58` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:28.519107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1b472ca74942c1acf8c47b35611f58)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:28.521699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:28.656551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:28.678767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1b472ca74942c1acf8c47b35611f58', 'request_id': 'req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:28.742711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1b472ca74942c1acf8c47b35611f58', 'request_id': 'req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:28.749289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1b472ca74942c1acf8c47b35611f58', 'request_id': 'req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:28.764710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] The token cache is being invalidate because project 7a1b472ca74942c1acf8c47b35611f58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:28.765213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:28.766057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa4dd05e-0f60-4fc6-ae72-2b12c88e38ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:28.774237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1044/2081] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:28 2022] DELETE /identity/v3/projects/7a1b472ca74942c1acf8c47b35611f58 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:28.776585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:28.778974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:28.779789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:28.853816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:28.855540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:28.931865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:28.955645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1038/2082] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:28.963355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:28.963947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:28.964452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:28.984185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:28.986153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:28.998987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1045/2083] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:28.999925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.010306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434d4be-4d07-4041-9440-23dc121fb9dc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.014936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.015550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.015836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.016107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.016763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:29.017069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.018932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.035241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1039/2084] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:35:29.037734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.048293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9776f-7601-4f4d-83da-a458484bd552 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.053413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.054021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.054299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.054600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.055285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.057189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.184956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1046/2085] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:29 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:29.189208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.199604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb1257-6e78-4b38-b10c-3d641f1f2906 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.260031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.260648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.260924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.261197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.261961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.263913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.355415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1040/2086] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:29 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:29.359731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.371599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb7a69-2886-4987-96a0-62a50b250622 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.376663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.377268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.377546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.377815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.378490 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.380297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.390583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1047/2087] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:29.394668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.404740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c15c8-1bc0-452f-af71-bbb05b5d6926 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.409973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19569858-fc8e-434b-9e25-8675801b8001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.410521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569858-fc8e-434b-9e25-8675801b8001 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.411448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569858-fc8e-434b-9e25-8675801b8001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.411836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19569858-fc8e-434b-9e25-8675801b8001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.412558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19569858-fc8e-434b-9e25-8675801b8001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.414226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19569858-fc8e-434b-9e25-8675801b8001 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.425269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1041/2088] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:35:29.429837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.440599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83da20-ebaf-4229-a1f3-bd86033e0fb3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.446402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.447100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.447381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.447654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] PATH_INFO: `/v3/projects/0873808364834af5931a68f0d8bb4089/users/43a60d4f7263472b81e2a2096683616c/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.462665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0873808364834af5931a68f0d8bb4089, user_id=43a60d4f7263472b81e2a2096683616c, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.465417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.543229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1048/2089] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:29 2022] PUT /identity/v3/projects/0873808364834af5931a68f0d8bb4089/users/43a60d4f7263472b81e2a2096683616c/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:29.555857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.556162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.556430 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.585335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] MFA Rules not processed for user `43a60d4f7263472b81e2a2096683616c`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:29.586690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:29.654445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1042/2090] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:29.659401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.727859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503d007-e895-48a3-ba5d-b899052e7e16 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.734302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.734986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.735279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.735547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.736227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.738167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:29.834670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1049/2091] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:29 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:29.835930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:29.848924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d48aa6-a2ae-4993-b825-471edb0c531c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:29.890134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:29.923153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:29.925208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:29.925682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:29.925962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:29.927344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:29.930415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.017657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1043/2092] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:29 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:30.022424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:30.034979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77672971-71f7-4241-9509-c75b85a0d520 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:30.041264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:30.042383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.042945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.043431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.044391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:30.046915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.064381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1050/2093] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:30.065208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:30.077054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e803b08-164f-436c-b632-99f9e5c1acf9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:30.084230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:30.084893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.085173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.085559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] PATH_INFO: `/v3/projects/4bebf4953ade4841893f2c0a603722ed/users/1156197d4dac419bb84a1acf582616de/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.110187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bebf4953ade4841893f2c0a603722ed, user_id=1156197d4dac419bb84a1acf582616de, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:30.113510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.196062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1044/2094] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:30 2022] PUT /identity/v3/projects/4bebf4953ade4841893f2c0a603722ed/users/1156197d4dac419bb84a1acf582616de/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:30.197734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:30.211958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c577ce-f189-4848-8c95-6b7aefb7237e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:30.245888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:30.295280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f9f8e-76d9-470b-9100-fc95d0b3a466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:30.295908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f9f8e-76d9-470b-9100-fc95d0b3a466 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.296194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f9f8e-76d9-470b-9100-fc95d0b3a466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.296493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f9f8e-76d9-470b-9100-fc95d0b3a466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.297120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f9f8e-76d9-470b-9100-fc95d0b3a466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:30.298814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f9f8e-76d9-470b-9100-fc95d0b3a466 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.314777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1051/2095] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:30.315280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:30.326963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e558b37-2a36-4b5b-b511-28a2f9d8a4ab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:30.333023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:30.334142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.334584 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.334961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] PATH_INFO: `/v3/projects/4bebf4953ade4841893f2c0a603722ed/users/1156197d4dac419bb84a1acf582616de/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.337038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bebf4953ade4841893f2c0a603722ed, user_id=1156197d4dac419bb84a1acf582616de, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:30.339944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.354394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1045/2096] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:30 2022] PUT /identity/v3/projects/4bebf4953ade4841893f2c0a603722ed/users/1156197d4dac419bb84a1acf582616de/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:30.365675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a329d5-3f41-4767-9381-065e706011c8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.366040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a329d5-3f41-4767-9381-065e706011c8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.366415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a329d5-3f41-4767-9381-065e706011c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.399448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d5a329d5-3f41-4767-9381-065e706011c8 None None] MFA Rules not processed for user `1156197d4dac419bb84a1acf582616de`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:30.400828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d5a329d5-3f41-4767-9381-065e706011c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:30.490917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1052/2097] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:30.500346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:30.516891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:30.602597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e816f1-cccc-4544-827f-f9732ccac9e9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.603204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e816f1-cccc-4544-827f-f9732ccac9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.603776 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e816f1-cccc-4544-827f-f9732ccac9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.603995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb634801-938a-4fd7-9982-a6df2aab74e4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:30.608313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:30.609010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.609343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.609646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.621165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:30.622779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.627106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-42e816f1-cccc-4544-827f-f9732ccac9e9 None None] MFA Rules not processed for user `2b9ab1983bb84318b509878b930e9402`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:30.629070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-42e816f1-cccc-4544-827f-f9732ccac9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:30.634290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1046/2098] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:35:30.696311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1053/2099] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:30.875168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:30.878180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:30.894130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34652a5-9277-4eaf-8ace-abf6ef3f9625 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:30.899692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:30.900303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:30.900883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:30.901158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:30.971623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:30.975228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:30.984797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1047/2100] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:33.121956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.121956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.122646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.150272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] MFA Rules not processed for user `b8349bd45ad84f14858f73bf42b48380`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:33.151465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:33.201435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:33.226385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1054/2101] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:33.231039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:33.236275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:33.276358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:33.313360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83865c92-59b2-4245-a664-179f05b70a95 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:33.318276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203937d-16a9-4424-b4a4-b28afb337b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:33.318939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203937d-16a9-4424-b4a4-b28afb337b30 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.319222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203937d-16a9-4424-b4a4-b28afb337b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.319490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203937d-16a9-4424-b4a4-b28afb337b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.320128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203937d-16a9-4424-b4a4-b28afb337b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:33.322064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203937d-16a9-4424-b4a4-b28afb337b30 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:33.460833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1048/2102] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:33 2022] POST /identity/v3/projects => generated 383 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:33.465047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:33.477600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce42547-260a-44b6-a3de-7efa9684fd12 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:33.565547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:33.565547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.565547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.566330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.566853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:33.569523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:33.571296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.572309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.573050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.596221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:33.597524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:33.664408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1049/2103] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:35:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:33.683337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1055/2104] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:33 2022] POST /identity/v3/users => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:35:33.686395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:33.727126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff663fa-899d-45ca-ada1-35ea38789cf5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:33.740927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:33.740927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.740927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.741646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.743605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:33.745496 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:33.756417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:33.758818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:33.783272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1050/2105] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:33.788503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:33.827623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2790c-6a47-4430-a596-4cebd25adb94 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:33.837314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:33.837964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.838250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.838560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] PATH_INFO: `/v3/projects/ef464bb9a5c44389851e147a8fd2866d/users/2882eb54386c48be8186b53ea3ce2857/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.856801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef464bb9a5c44389851e147a8fd2866d, user_id=2882eb54386c48be8186b53ea3ce2857, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:33.859575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:33.909929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d740b-9ac0-4274-a68a-85b77d40108e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:33.915839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:33.916475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:33.916824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:33.917141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:33.928011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:33.932122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:33.952550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1056/2106] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:35:34.016579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1051/2107] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:33 2022] PUT /identity/v3/projects/ef464bb9a5c44389851e147a8fd2866d/users/2882eb54386c48be8186b53ea3ce2857/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:34.023766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:34.063098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3da96-d212-4d4a-af41-a4227f6fd538 service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:34.230348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:34.231017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:34.231272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:34.231938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:34.232920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:34.235038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:34.239289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:34.254294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1057/2108] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:34.255817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:34.272428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3889-b061-4073-a38c-11fbd4c27fb6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:34.292360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bb347-eb27-4323-9208-b154ab8b1900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:34.293050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb347-eb27-4323-9208-b154ab8b1900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:34.293422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb347-eb27-4323-9208-b154ab8b1900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:34.293637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bb347-eb27-4323-9208-b154ab8b1900 admin admin] PATH_INFO: `/v3/projects/ef464bb9a5c44389851e147a8fd2866d/users/2882eb54386c48be8186b53ea3ce2857/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:34.295607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bb347-eb27-4323-9208-b154ab8b1900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef464bb9a5c44389851e147a8fd2866d, user_id=2882eb54386c48be8186b53ea3ce2857, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:34.298727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bb347-eb27-4323-9208-b154ab8b1900 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:34.342760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1052/2109] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:34 2022] PUT /identity/v3/projects/ef464bb9a5c44389851e147a8fd2866d/users/2882eb54386c48be8186b53ea3ce2857/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:39.775584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:39.788958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd5c6d-e279-4e9c-9eba-afec288f2f22 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:39.887879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:39.888537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:39.890381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:39.890737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] PATH_INFO: `/v3/users/43a60d4f7263472b81e2a2096683616c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:39.892241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a60d4f7263472b81e2a2096683616c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:39.894316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:40.189908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a60d4f7263472b81e2a2096683616c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:40.330976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a60d4f7263472b81e2a2096683616c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:40.331723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a60d4f7263472b81e2a2096683616c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:40.365791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a60d4f7263472b81e2a2096683616c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:40.377658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a60d4f7263472b81e2a2096683616c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:40.411337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1058/2110] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:39 2022] DELETE /identity/v3/users/43a60d4f7263472b81e2a2096683616c => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:40.415453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:40.415453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:40.415453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:40.475113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:40.480926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:40.572541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1053/2111] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:40.585869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:40.585869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:40.699493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f8d63-c31e-4ed8-9491-a030f7a52513 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:40.704091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:40.704719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:40.705291 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:40.705680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:40.720866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:40.722500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:40.737290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1059/2112] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:41.514888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:41.518386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:41.536959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d0c65-ca7b-4938-a7b3-0b1321be26e9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:41.542504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:41.543148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:41.544070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:41.544651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] PATH_INFO: `/v3/projects/0873808364834af5931a68f0d8bb4089` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:41.545816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0873808364834af5931a68f0d8bb4089)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:41.548351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:41.561451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:41.689637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0873808364834af5931a68f0d8bb4089', 'request_id': 'req-a779f239-e949-4a28-bbfc-9208620517cb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:41.769414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0873808364834af5931a68f0d8bb4089', 'request_id': 'req-a779f239-e949-4a28-bbfc-9208620517cb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:41.779336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0873808364834af5931a68f0d8bb4089', 'request_id': 'req-a779f239-e949-4a28-bbfc-9208620517cb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:41.811011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] The token cache is being invalidate because project 0873808364834af5931a68f0d8bb4089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:41.812334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:41.812334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:41.813888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1054/2113] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:41 2022] DELETE /identity/v3/projects/0873808364834af5931a68f0d8bb4089 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:41.818096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:41.840183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035b28-4941-4470-9900-d50f9680d94e service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:41.888106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:41.969840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:41.970511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:41.970797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:41.971071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] PATH_INFO: `/v3/users/1156197d4dac419bb84a1acf582616de` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:41.974586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1156197d4dac419bb84a1acf582616de)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:41.980225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:42.243067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1156197d4dac419bb84a1acf582616de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:42.313875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1156197d4dac419bb84a1acf582616de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:42.314625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1156197d4dac419bb84a1acf582616de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:42.338900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1156197d4dac419bb84a1acf582616de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:42.392910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fcfd5f7d-0c78-49ac-a5da-64fde5ba9c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1156197d4dac419bb84a1acf582616de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:42.403765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1060/2114] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:41 2022] DELETE /identity/v3/users/1156197d4dac419bb84a1acf582616de => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:43.390724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:43.392174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:43.420179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a779f239-e949-4a28-bbfc-9208620517cb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:43.508659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:43.509851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:43.510294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:43.510760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] PATH_INFO: `/v3/projects/4bebf4953ade4841893f2c0a603722ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:43.512413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bebf4953ade4841893f2c0a603722ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:43.518404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:43.814717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bebf4953ade4841893f2c0a603722ed', 'request_id': 'req-71b60260-c912-497b-92f6-280cd00b3d80'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:43.892466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bebf4953ade4841893f2c0a603722ed', 'request_id': 'req-71b60260-c912-497b-92f6-280cd00b3d80'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:43.912826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bebf4953ade4841893f2c0a603722ed', 'request_id': 'req-71b60260-c912-497b-92f6-280cd00b3d80'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:43.951177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] The token cache is being invalidate because project 4bebf4953ade4841893f2c0a603722ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:43.951683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:43.952380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b60260-c912-497b-92f6-280cd00b3d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:43.960784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1055/2115] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:43 2022] DELETE /identity/v3/projects/4bebf4953ade4841893f2c0a603722ed => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:43.962618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:43.963404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:43.963838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.007297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:44.008915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:44.170085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1061/2116] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:44.179683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:44.180082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:44.180428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.228962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:44.232346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:44.255517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1056/2117] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:44.257849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:44.269565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3874f-193e-4281-ab7f-b3e1bd22e0b5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:44.294699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:44.295951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:44.296348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:44.296625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.297308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:44.297620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:44.300070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:44.354014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1062/2118] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:35:44.355406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:44.373606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca0cc-4a5d-4df0-8f89-508b98ab0f6a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:44.386118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:44.389005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:44.389833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:44.391851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.393198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:44.396072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:44.507076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:44.526248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1057/2119] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:44 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:44.530219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:44.541431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db35e6c-ed7b-4b4a-9699-d4769a8adb5d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:44.604389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:44.677196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:44.678379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:44.678862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:44.679337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.680229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:44.682772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:44.821872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1063/2120] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:44 2022] POST /identity/v3/users => generated 287 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:44.825722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:44.838199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ce8c-8c79-4bfa-9d25-29f60d32af56 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:44.848985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:44.849736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:44.850026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:44.850302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.851869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:44.855364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:44.886015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1058/2121] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:44.901976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:44.938729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb761-b981-4a29-ab8c-9591bb25362d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:44.944331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e31bd-3e00-4fa7-846c-752e2073b801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:44.944954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e31bd-3e00-4fa7-846c-752e2073b801 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:44.945240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e31bd-3e00-4fa7-846c-752e2073b801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:44.945517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e31bd-3e00-4fa7-846c-752e2073b801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:44.946243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e31bd-3e00-4fa7-846c-752e2073b801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:44.947913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e31bd-3e00-4fa7-846c-752e2073b801 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:44.966759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1064/2122] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:44.972194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:44.992081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ee519-6318-4c65-842d-d327f799f8f8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:45.004421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:45.008018 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:45.008347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:45.008629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] PATH_INFO: `/v3/projects/e11c2051617c430d851dbc62477195c2/users/bfb4e9d9d4c24b509f51485c648a1f95/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:45.086535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11c2051617c430d851dbc62477195c2, user_id=bfb4e9d9d4c24b509f51485c648a1f95, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:45.107644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:45.344431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1059/2123] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:44 2022] PUT /identity/v3/projects/e11c2051617c430d851dbc62477195c2/users/bfb4e9d9d4c24b509f51485c648a1f95/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:45.354577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bb3c8-3f5c-454f-b148-19fe5aad7b3b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:45.354577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bb3c8-3f5c-454f-b148-19fe5aad7b3b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:45.354577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bb3c8-3f5c-454f-b148-19fe5aad7b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:45.392507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a59bb3c8-3f5c-454f-b148-19fe5aad7b3b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:45.394567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a59bb3c8-3f5c-454f-b148-19fe5aad7b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:45.527576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1065/2124] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:45.541656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:45.543566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:45.695878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da850678-68ad-4836-a5bd-137065cae963 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:45.700051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:45.701980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:45.702914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:45.704404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:45.739509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:45.743268 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:45.760749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1060/2125] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:46.458750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de075dc-efe4-4fa2-83f2-f5aee3d0d09c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:46.458750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de075dc-efe4-4fa2-83f2-f5aee3d0d09c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:46.460784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de075dc-efe4-4fa2-83f2-f5aee3d0d09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:46.507823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6de075dc-efe4-4fa2-83f2-f5aee3d0d09c None None] MFA Rules not processed for user `2882eb54386c48be8186b53ea3ce2857`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:46.509696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6de075dc-efe4-4fa2-83f2-f5aee3d0d09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:46.722395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1066/2126] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:46.750536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:46.752604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:46.868480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec578b-0511-464b-a006-0a4d8df5454a service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:46.884437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:46.887720 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:46.888143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:46.888651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:46.893301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:47.064615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:47.067981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:47.082624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1061/2127] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:47.591394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4e2f2-c8a8-4ddd-95a4-5178531dbf42 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:47.592375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4e2f2-c8a8-4ddd-95a4-5178531dbf42 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:47.592856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4e2f2-c8a8-4ddd-95a4-5178531dbf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:47.637125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c3c4e2f2-c8a8-4ddd-95a4-5178531dbf42 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:47.639002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c3c4e2f2-c8a8-4ddd-95a4-5178531dbf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:47.684117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:47.757774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1067/2128] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:35:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:47.804516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:47.806032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:47.866797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:47.931871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e900351-8c86-41d9-a7d2-eccfe4384480 service glance] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:47.938509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:47.939332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:47.939777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:47.940431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:47.954309 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:47.957071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:48.000168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1062/2129] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:35:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:35:48.105521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:48.105807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:48.106315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:48.178908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] MFA Rules not processed for user `bfb4e9d9d4c24b509f51485c648a1f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:48.180286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:48.365610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1068/2130] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:48.366966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:48.410072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098e99-b780-4988-92fe-0746357313c2 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:48.437159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:48.438233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:48.438736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:48.442586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:48.442586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:48.442586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:48.612236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1063/2131] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:48 2022] POST /identity/v3/projects => generated 371 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:48.618080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:48.638007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bf53c-6e67-4d93-895d-828f4bd0291e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:48.813843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:48.817010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:48.817010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:48.817010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:48.817010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:48.825517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:48.928864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1069/2132] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:48 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:48.932780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:48.963050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae221438-c461-4280-9a37-afb7eb3c27fc admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:48.995978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:48.998199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:48.998527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:49.003588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:49.003764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:49.009815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:49.039390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1064/2133] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:49.043340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:49.063854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa957a-3769-4e08-972a-862213d6cfe9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:49.085071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:49.085775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:49.086058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:49.086330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] PATH_INFO: `/v3/projects/e95943d2e5c649029f298a504fa03a6a/users/4fbe75e4d73740f4a01e6e1130bc0871/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:49.127072 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95943d2e5c649029f298a504fa03a6a, user_id=4fbe75e4d73740f4a01e6e1130bc0871, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:49.130136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:49.269269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1070/2134] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:49 2022] PUT /identity/v3/projects/e95943d2e5c649029f298a504fa03a6a/users/4fbe75e4d73740f4a01e6e1130bc0871/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:49.275358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:49.295884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ee773-899d-430d-af5c-a3a2dd974473 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:49.715669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:49.718669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:49.718956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:49.719232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:49.727478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:49.732772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:49.773329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1065/2135] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:49.779284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:49.817230 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578694-7af7-404b-bacf-289abf501869 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:49.847021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:49.849543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:49.853062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b28369-1329-4c53-854c-cc958f12f6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:49.865360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b28369-1329-4c53-854c-cc958f12f6df admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:49.868690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b28369-1329-4c53-854c-cc958f12f6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:49.869222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b28369-1329-4c53-854c-cc958f12f6df admin admin] PATH_INFO: `/v3/projects/e95943d2e5c649029f298a504fa03a6a/users/4fbe75e4d73740f4a01e6e1130bc0871/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:49.876186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d805390-5f60-40dd-92f3-bbbcdf408922 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:49.882840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b28369-1329-4c53-854c-cc958f12f6df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95943d2e5c649029f298a504fa03a6a, user_id=4fbe75e4d73740f4a01e6e1130bc0871, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:49.885979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b28369-1329-4c53-854c-cc958f12f6df admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:49.898716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:49.899866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:49.899866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:49.900028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] PATH_INFO: `/v3/users/5c3a7ab81e3a4c3dbc7d7234fbb3ba5a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:49.911237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3a7ab81e3a4c3dbc7d7234fbb3ba5a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:49.921600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:49.930766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1071/2136] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:49 2022] PUT /identity/v3/projects/e95943d2e5c649029f298a504fa03a6a/users/4fbe75e4d73740f4a01e6e1130bc0871/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:50.227284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3a7ab81e3a4c3dbc7d7234fbb3ba5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:50.232656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3a7ab81e3a4c3dbc7d7234fbb3ba5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:50.285288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3a7ab81e3a4c3dbc7d7234fbb3ba5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:50.301895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:50.311349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3a7ab81e3a4c3dbc7d7234fbb3ba5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:50.404808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3a7ab81e3a4c3dbc7d7234fbb3ba5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:50.416527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1066/2137] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:35:49 2022] DELETE /identity/v3/users/5c3a7ab81e3a4c3dbc7d7234fbb3ba5a => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:35:50.418157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:50.418828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:50.419375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:50.451432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:50.460311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:50.490466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:50.618187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1072/2138] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:50.650382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:50.652348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:50.817928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4130186-915a-4bf9-9cb5-c7bd90704322 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:50.833842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7870f-b96d-46a7-830a-2f699e3cd8be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:50.838164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7870f-b96d-46a7-830a-2f699e3cd8be service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:50.838486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7870f-b96d-46a7-830a-2f699e3cd8be service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:50.838768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7870f-b96d-46a7-830a-2f699e3cd8be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:50.875313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7870f-b96d-46a7-830a-2f699e3cd8be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:50.878679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7870f-b96d-46a7-830a-2f699e3cd8be service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:50.909141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1067/2139] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:52.117761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:52.119885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:52.164191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5b17-c233-4e0a-b55c-1709f2797051 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:52.206114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:52.209983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:52.210843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:52.213472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] PATH_INFO: `/v3/projects/d9ad6a82910044ca9d6a287d687cac8b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:52.228213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ad6a82910044ca9d6a287d687cac8b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:52.236190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:52.544296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ad6a82910044ca9d6a287d687cac8b', 'request_id': 'req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:52.645154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ad6a82910044ca9d6a287d687cac8b', 'request_id': 'req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:52.662742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ad6a82910044ca9d6a287d687cac8b', 'request_id': 'req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:52.706015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] The token cache is being invalidate because project d9ad6a82910044ca9d6a287d687cac8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:35:52.706313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:52.712570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27b707fc-44c2-4bbf-9b8b-e691a0f7f104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:35:52.718836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1073/2140] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:35:52 2022] DELETE /identity/v3/projects/d9ad6a82910044ca9d6a287d687cac8b => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:52.724548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40427350-c564-472e-b353-81024e76054b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:52.725536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:52.725840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:52.726098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40427350-c564-472e-b353-81024e76054b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:52.726233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:52.729420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40427350-c564-472e-b353-81024e76054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:52.770298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-40427350-c564-472e-b353-81024e76054b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:52.771653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-40427350-c564-472e-b353-81024e76054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:52.826716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] MFA Rules not processed for user `4fbe75e4d73740f4a01e6e1130bc0871`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:52.826716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:52.965642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1068/2141] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:52.967319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:52.967611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:52.967892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:53.010927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:53.012182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:53.035371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1069/2142] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:53.040673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:53.072308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519aa26-0625-4168-be02-8811439b1c64 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:53.088445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1074/2143] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:35:53.093497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:53.094450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:53.094967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:53.095456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:53.096880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:35:53.097495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:53.097905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:53.099510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:53.103239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:53.142449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1070/2144] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:35:53.151292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:53.160728 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:53.195875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d16410-a973-42c6-9a05-8946e4eb58ac admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:53.255947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:53.257009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:53.257695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:53.257934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea6965-bc2d-42a1-979b-c0fcf0c75519 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:53.258535 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:53.261162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:53.262248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:53.262992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:53.263398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:53.263720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:53.269951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:53.277577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:53.376005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:53.378036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:53.403498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1075/2145] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:35:53.656941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1071/2146] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:35:53 2022] POST /identity/v3/projects => generated 363 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:53.662747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:53.697398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6543fa-d363-4f3b-813c-c1d4ca4fe6e9 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:53.853189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:53.854075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:53.855803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:53.856736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:53.859827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:53.862755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:53.993951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1076/2147] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:35:53 2022] POST /identity/v3/users => generated 282 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:35:53.998267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:54.028711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cfb20-1bcf-4a03-8fcc-33a6565ca5be admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:54.062035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:54.063204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:54.066782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:54.067740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:54.073499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:54.076047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:54.112833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1072/2148] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:54.119834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:54.141568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e85fc-b7f8-41e8-9964-20fbc615322d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:54.151556 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbef78-095c-493e-b16d-d8ad914195d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:54.152200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbef78-095c-493e-b16d-d8ad914195d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:54.152484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbef78-095c-493e-b16d-d8ad914195d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:54.152758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbef78-095c-493e-b16d-d8ad914195d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:54.153416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbef78-095c-493e-b16d-d8ad914195d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:54.155074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbef78-095c-493e-b16d-d8ad914195d7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:54.194183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1077/2149] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:35:54.198238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:54.213374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ee7a9-8dce-4e2d-8f1a-3b3150af2f6c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:54.228383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:54.230897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:54.230897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:54.230897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] PATH_INFO: `/v3/projects/9fcf666e7f704bfd8817469bfd76a933/users/f6c0c784e29444d395e9814f204fd26e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:54.271181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcf666e7f704bfd8817469bfd76a933, user_id=f6c0c784e29444d395e9814f204fd26e, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:54.274511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:54.426681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1073/2150] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:35:54 2022] PUT /identity/v3/projects/9fcf666e7f704bfd8817469bfd76a933/users/f6c0c784e29444d395e9814f204fd26e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:35:54.432425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:54.433089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:54.433366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:54.505517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] MFA Rules not processed for user `f6c0c784e29444d395e9814f204fd26e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:35:54.507332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:54.740464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:54.806673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1078/2151] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:35:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:35:54.819156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:54.850817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:35:54.978711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2a2ed-6875-48fb-bcf8-40d522a740a5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:54.991926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48271a13-7398-4370-baba-503f78266592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:54.991926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48271a13-7398-4370-baba-503f78266592 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:54.991926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48271a13-7398-4370-baba-503f78266592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:54.992363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48271a13-7398-4370-baba-503f78266592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:55.017779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48271a13-7398-4370-baba-503f78266592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:55.022316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48271a13-7398-4370-baba-503f78266592 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:55.045875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1074/2152] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:35:55.601408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:35:55.603831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:35:55.629903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfee3b-9563-4d9d-a9c2-daae6d4c2f58 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:35:55.649207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b8a8-cbae-4b37-8fa5-47ea8926ca32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:35:55.650388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b8a8-cbae-4b37-8fa5-47ea8926ca32 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:35:55.654481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b8a8-cbae-4b37-8fa5-47ea8926ca32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:35:55.654481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b8a8-cbae-4b37-8fa5-47ea8926ca32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:35:55.815323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b8a8-cbae-4b37-8fa5-47ea8926ca32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:35:55.829245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b8a8-cbae-4b37-8fa5-47ea8926ca32 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:35:55.848316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1079/2153] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:35:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:36:00.967941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8cb1-fa3f-4d46-ada5-68056ad35337 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:00.968473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8cb1-fa3f-4d46-ada5-68056ad35337 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:00.968960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d8cb1-fa3f-4d46-ada5-68056ad35337 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:00.971502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1075/2154] 10.222.0.13 () {54 vars in 961 bytes} [Sat Sep 3 01:36:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:36:00.978582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd581368-efe0-419f-b1b7-16d045ba2f04 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:00.978582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd581368-efe0-419f-b1b7-16d045ba2f04 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:00.979755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd581368-efe0-419f-b1b7-16d045ba2f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:01.027546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cd581368-efe0-419f-b1b7-16d045ba2f04 None None] MFA Rules not processed for user `54b10e8ec2e244898f250cb987b6c7c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:01.028868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cd581368-efe0-419f-b1b7-16d045ba2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:01.183289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1080/2155] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Sep 3 01:36:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:36:01.193453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d8cb1-fa3f-4d46-ada5-68056ad35337 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:01.215537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d8cb1-fa3f-4d46-ada5-68056ad35337 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:01.229854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:01.230747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:01.231200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:01.231677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:01.236830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:01.263374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:01.267038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:01.297591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1076/2156] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:03.527298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2e6b6-6f6e-4a73-952a-3ca4795ca7e0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:03.531838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2e6b6-6f6e-4a73-952a-3ca4795ca7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:03.532551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2e6b6-6f6e-4a73-952a-3ca4795ca7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:03.578503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-add2e6b6-6f6e-4a73-952a-3ca4795ca7e0 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:03.581108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-add2e6b6-6f6e-4a73-952a-3ca4795ca7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:03.715710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1081/2157] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:36:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:03.732960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:03.735063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:03.837570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720544eb-0be9-4f32-8def-a925d7fe938b service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:03.844370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:03.845011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:03.845468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:03.847288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:03.869051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:03.870805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:03.881187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1077/2158] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:36:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 01:36:05.649577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d1e3-ac89-41eb-9adb-c99f196ce626 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:05.650147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d1e3-ac89-41eb-9adb-c99f196ce626 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:05.650147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d1e3-ac89-41eb-9adb-c99f196ce626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:05.673022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0f45d1e3-ac89-41eb-9adb-c99f196ce626 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:05.683815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0f45d1e3-ac89-41eb-9adb-c99f196ce626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:05.696421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1082/2159] 10.222.0.13 () {58 vars in 967 bytes} [Sat Sep 3 01:36:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:05.708883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:05.728653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064f93c-5412-4a47-a0f2-dbb87d600b33 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:05.735148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb161f-f565-4e63-bf6a-ed68472bc80e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:05.736278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb161f-f565-4e63-bf6a-ed68472bc80e service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:05.736751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb161f-f565-4e63-bf6a-ed68472bc80e service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:05.737236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb161f-f565-4e63-bf6a-ed68472bc80e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:05.750994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb161f-f565-4e63-bf6a-ed68472bc80e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:05.756693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb161f-f565-4e63-bf6a-ed68472bc80e service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:05.777635 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1078/2160] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:36:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:36:05.801157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:05.801705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:05.802234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:05.841265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] MFA Rules not processed for user `b8349bd45ad84f14858f73bf42b48380`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:05.843089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:05.863246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:05.974748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1083/2161] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:12.215840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:12.216444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:12.216444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:12.245158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:12.245158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:12.257821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1079/2162] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:12.264829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:12.289420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1f571-a682-4bfa-98ac-63de81280ab6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:12.309354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:12.310090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:12.314678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:12.317381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:12.340369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:12.342573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:12.365975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1084/2163] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:36:12.805118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:12.807115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:13.048271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b124d-2858-4093-9e23-b97fb0914d2b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:13.062954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:13.063653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:13.063954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:13.064288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] PATH_INFO: `/v3/users/205b8f60af7a4957a578f21f9ea464dd` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:13.070480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205b8f60af7a4957a578f21f9ea464dd)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:13.079344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:13.118323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:13.419598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205b8f60af7a4957a578f21f9ea464dd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:13.420122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205b8f60af7a4957a578f21f9ea464dd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:13.496750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205b8f60af7a4957a578f21f9ea464dd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:13.520352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205b8f60af7a4957a578f21f9ea464dd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:13.648177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c2209857-0416-4132-98ee-6dfc3b0fdd54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205b8f60af7a4957a578f21f9ea464dd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:13.669356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1080/2164] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:12 2022] DELETE /identity/v3/users/205b8f60af7a4957a578f21f9ea464dd => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:14.140418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:14.162819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4b6cd-2312-42d5-a6cb-79afeaca1c68 service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:14.399190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:14.399190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:14.399190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:14.399190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] PATH_INFO: `/v3/projects/efd244ba290249568e3a6f8bd3e7e814` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:14.399962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd244ba290249568e3a6f8bd3e7e814)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:14.399962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:14.876704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd244ba290249568e3a6f8bd3e7e814', 'request_id': 'req-6591918d-7620-4515-8c6f-76a12c58692f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:14.943264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd244ba290249568e3a6f8bd3e7e814', 'request_id': 'req-6591918d-7620-4515-8c6f-76a12c58692f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:14.953002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd244ba290249568e3a6f8bd3e7e814', 'request_id': 'req-6591918d-7620-4515-8c6f-76a12c58692f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:14.972895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] The token cache is being invalidate because project efd244ba290249568e3a6f8bd3e7e814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:14.975914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:14.975914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6591918d-7620-4515-8c6f-76a12c58692f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:14.983338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1085/2165] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:14 2022] DELETE /identity/v3/projects/efd244ba290249568e3a6f8bd3e7e814 => generated 0 bytes in 841 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:14.985215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:14.986667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:14.987159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:15.049772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:15.053218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:15.234223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1081/2166] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:15.243592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:15.244073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:15.244073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:15.265334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:15.266610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:15.278114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1086/2167] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:15.279440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:15.291052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed056c-f054-4104-929b-7b7c434a039e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:15.296361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:15.296974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:15.297416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:15.297849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:15.298634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:36:15.299114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:15.301451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:15.319918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1082/2168] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:36:15.321401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:15.331691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e181c3-e2fa-4912-abd1-d3016f982712 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:15.337240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:15.337852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:15.338134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:15.338456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:15.339126 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:15.341355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:15.573227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1087/2169] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:36:15 2022] POST /identity/v3/projects => generated 391 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:15.577483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:15.592997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3d1ba-7504-41d0-9d36-c74064be05e2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:15.719956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:15.720716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:15.721005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:15.721339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:15.722025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:15.731528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:15.901178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1083/2170] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:36:15 2022] POST /identity/v3/users => generated 296 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:15.904449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:15.928204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb0e50-5a28-4c32-a390-40e1ba674664 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:15.942443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:15.947533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:15.948725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:15.949770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:15.952398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:15.967183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:16.015428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:16.029012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1088/2171] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:16.033933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:16.050167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c2ce7e-55aa-48a8-8964-7b46a7a9aa6d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:16.069981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e8d30-a00d-41e5-b15c-bfdfdd903d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:16.070542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e8d30-a00d-41e5-b15c-bfdfdd903d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:16.070824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e8d30-a00d-41e5-b15c-bfdfdd903d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:16.071097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e8d30-a00d-41e5-b15c-bfdfdd903d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:16.071754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e8d30-a00d-41e5-b15c-bfdfdd903d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:16.073382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e8d30-a00d-41e5-b15c-bfdfdd903d2e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:16.086401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:16.104970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1084/2172] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:16.111304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:16.130036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928b09-ea57-4247-b5e7-3c1e50c41185 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:16.135838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:16.136681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:16.137088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:16.137474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] PATH_INFO: `/v3/projects/174605548d884b8c89e2739692609de2/users/feba07a3c1bd42dfa56780062ea7aae3/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:16.166301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=174605548d884b8c89e2739692609de2, user_id=feba07a3c1bd42dfa56780062ea7aae3, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:16.169484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:16.304620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1089/2173] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:16 2022] PUT /identity/v3/projects/174605548d884b8c89e2739692609de2/users/feba07a3c1bd42dfa56780062ea7aae3/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:16.317296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:16.328244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:16.328244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:16.403363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] MFA Rules not processed for user `feba07a3c1bd42dfa56780062ea7aae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:16.404790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:16.574650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1085/2174] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:16.586522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:16.589669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:16.737613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad11d-6267-4833-b4be-87b4518b8d48 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:16.755982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:16.755982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:16.755982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:16.755982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:16.756684 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:16.757164 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:16.771787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1090/2175] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:20.823923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:20.825995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:20.934173 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381e08d-c23d-44db-89ea-f3716c27d763 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:20.939461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:20.940109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:20.940392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:20.940662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] PATH_INFO: `/v3/users/bfb4e9d9d4c24b509f51485c648a1f95` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:20.942497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb4e9d9d4c24b509f51485c648a1f95)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:20.944946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:21.227951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb4e9d9d4c24b509f51485c648a1f95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:21.228687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb4e9d9d4c24b509f51485c648a1f95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:21.247693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb4e9d9d4c24b509f51485c648a1f95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:21.268102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb4e9d9d4c24b509f51485c648a1f95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:21.340884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb4e9d9d4c24b509f51485c648a1f95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:21.347661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1086/2176] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:20 2022] DELETE /identity/v3/users/bfb4e9d9d4c24b509f51485c648a1f95 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:36:21.619461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:21.637898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29ad86-1c88-4eed-80e3-18988123fb8c service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:21.726045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:21.726814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:21.727200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:21.727587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] PATH_INFO: `/v3/projects/e11c2051617c430d851dbc62477195c2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:21.728900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11c2051617c430d851dbc62477195c2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:21.731399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:21.935324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11c2051617c430d851dbc62477195c2', 'request_id': 'req-f57099e7-eba7-4fe6-b63e-20588f48e816'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.051060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11c2051617c430d851dbc62477195c2', 'request_id': 'req-f57099e7-eba7-4fe6-b63e-20588f48e816'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.059292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11c2051617c430d851dbc62477195c2', 'request_id': 'req-f57099e7-eba7-4fe6-b63e-20588f48e816'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.074509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] The token cache is being invalidate because project e11c2051617c430d851dbc62477195c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:22.075045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.075751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.077342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1091/2177] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:21 2022] DELETE /identity/v3/projects/e11c2051617c430d851dbc62477195c2 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:22.083150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:22.084658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:22.100027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8055fa-672b-47dc-9af0-9f4c7e85058c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:22.104343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:22.168760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:22.169637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:22.169987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:22.170481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] PATH_INFO: `/v3/users/4fbe75e4d73740f4a01e6e1130bc0871` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:22.172073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbe75e4d73740f4a01e6e1130bc0871)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:22.174848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:22.426881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbe75e4d73740f4a01e6e1130bc0871'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.427566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbe75e4d73740f4a01e6e1130bc0871'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.461361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbe75e4d73740f4a01e6e1130bc0871'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.468941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbe75e4d73740f4a01e6e1130bc0871'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.551034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78f85edd-7cf3-4862-9340-dda2f589206f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbe75e4d73740f4a01e6e1130bc0871'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:22.564053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1087/2178] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:22 2022] DELETE /identity/v3/users/4fbe75e4d73740f4a01e6e1130bc0871 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:22.818708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:22.829777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:22.846268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57099e7-eba7-4fe6-b63e-20588f48e816 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:22.857273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:22.935266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:22.935948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:22.936347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:22.936500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] PATH_INFO: `/v3/projects/e95943d2e5c649029f298a504fa03a6a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:22.937745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95943d2e5c649029f298a504fa03a6a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:22.939895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:23.042964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95943d2e5c649029f298a504fa03a6a', 'request_id': 'req-9e8cae99-10b3-4907-80e7-616255d5962d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:23.115404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95943d2e5c649029f298a504fa03a6a', 'request_id': 'req-9e8cae99-10b3-4907-80e7-616255d5962d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:23.134100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95943d2e5c649029f298a504fa03a6a', 'request_id': 'req-9e8cae99-10b3-4907-80e7-616255d5962d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:23.147238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] The token cache is being invalidate because project e95943d2e5c649029f298a504fa03a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:23.147569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:23.148193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9e8cae99-10b3-4907-80e7-616255d5962d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:23.155143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1092/2179] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:22 2022] DELETE /identity/v3/projects/e95943d2e5c649029f298a504fa03a6a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:23.160828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.161367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.161367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.183100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:23.184953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:23.264488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1088/2180] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:36:23.274149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.274464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.274759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.295988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:23.297834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:23.307570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1093/2181] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:23.314129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:23.326264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e1967-81c2-4a39-a831-41a692367601 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:23.332846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:23.333486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.333893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.334308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.336034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:36:23.336151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:23.339075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:23.353680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1089/2182] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:36:23.359516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:23.376027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e729a7d-f9ab-478e-a1bc-5d6b062c42a4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:23.385343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:23.386682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.387371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.387877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.388570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:23.390594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:23.477430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1094/2183] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:36:23 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:23.487601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:23.503111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e8c8-12da-4e88-bacc-bd89871c00c1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:23.517700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:23.624808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:23.625754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.626399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.626855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.627685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:23.630083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:23.707251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1090/2184] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:36:23 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:23.707527 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:23.730802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e925c2c-bd85-4eac-ad2a-55db9bae6f72 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:23.761859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:23.763035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.763515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.766769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.766769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:23.766769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:23.790792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1095/2185] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:36:23.794798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:23.837928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec70d4-f80a-4ac8-9387-684b19acbd6b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:23.843613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871ff0-9770-4adb-acd0-c7f397f05132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:23.844240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871ff0-9770-4adb-acd0-c7f397f05132 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.844524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871ff0-9770-4adb-acd0-c7f397f05132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.844820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871ff0-9770-4adb-acd0-c7f397f05132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.845610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871ff0-9770-4adb-acd0-c7f397f05132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:23.847634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871ff0-9770-4adb-acd0-c7f397f05132 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:23.865059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1091/2186] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:36:23.870831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:23.897309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b4d42-6906-4793-9f23-43c89b9b9046 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:23.903150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:23.903902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:23.904218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:23.904533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] PATH_INFO: `/v3/projects/39add67fc96f48e4a59198ab7e815c21/users/7c72e514966a43848524c53e5f61680e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:23.938111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39add67fc96f48e4a59198ab7e815c21, user_id=7c72e514966a43848524c53e5f61680e, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:23.945387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:24.070808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1096/2187] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:23 2022] PUT /identity/v3/projects/39add67fc96f48e4a59198ab7e815c21/users/7c72e514966a43848524c53e5f61680e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:36:24.082031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:24.082816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:24.083016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:24.207845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] MFA Rules not processed for user `7c72e514966a43848524c53e5f61680e`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:24.207845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:24.387355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1092/2188] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:24.393582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:24.504628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:24.543563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0a197-40d4-4c66-90cf-31115b2614c3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:24.548529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:24.549149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:24.549430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:24.549701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:24.550432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:24.557131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:25.440536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1097/2189] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:36:24 2022] POST /identity/v3/projects => generated 385 bytes in 1048 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:25.445354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:25.458956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea1763-ce9c-431f-bc62-7693c814ca00 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:25.535351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:25.536148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:25.536440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:25.536723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:25.537381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:25.539609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:25.702087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1093/2190] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:36:25 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:25.706143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:25.717248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f3fc7-682b-4283-a7e5-cc6c06c0aac4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:25.721828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:25.722764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:25.722986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:25.723240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:25.723902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:25.725509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:25.735819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1098/2191] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:25.739908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:25.751338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d68685-0971-447b-a1b8-c8de301b872a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:25.756585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c2875-b462-4c62-8816-d705d7c92eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:25.757194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c2875-b462-4c62-8816-d705d7c92eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:25.757469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c2875-b462-4c62-8816-d705d7c92eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:25.757736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c2875-b462-4c62-8816-d705d7c92eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:25.758473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c2875-b462-4c62-8816-d705d7c92eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:25.760127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c2875-b462-4c62-8816-d705d7c92eb7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:25.765766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:25.771757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1094/2192] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:25.776233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:25.787511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd69de-1625-48f7-905a-68b100220bdd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:25.792311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:25.792970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:25.793246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:25.793531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] PATH_INFO: `/v3/projects/1e5fa480a2364906a5bb0585ef20adf2/users/b7a0b9c587e14232bb0ad476c4b390ae/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:25.812108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5fa480a2364906a5bb0585ef20adf2, user_id=b7a0b9c587e14232bb0ad476c4b390ae, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:25.813270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:25.886762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1099/2193] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:25 2022] PUT /identity/v3/projects/1e5fa480a2364906a5bb0585ef20adf2/users/b7a0b9c587e14232bb0ad476c4b390ae/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:25.897155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:25.897447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:25.897752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:25.927511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] MFA Rules not processed for user `b7a0b9c587e14232bb0ad476c4b390ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:25.928763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:25.996150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1095/2194] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:26.000641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.064910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2d580-fbf7-4daa-8655-b664a73719a1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.070645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.071297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.071610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.071884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.072529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:26.074574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:26.189465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1100/2195] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:36:25 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:36:26.198619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.209831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564ba58-55a4-4eb8-8cd1-1b8581c48ecb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.274651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.275264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.275541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.275810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.276450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:26.278435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:26.366796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1096/2196] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:36:26 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:26.370892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.381817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ae42f-d18f-42ef-b274-9d61ab2d1e51 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.387275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.389363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.389363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.389363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.389363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:26.391112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:26.403410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1101/2197] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:26.407882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.418833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca298e-6c2d-4f40-9ae7-e5488c5772bf admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.423743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.424400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.424715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.424991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] PATH_INFO: `/v3/projects/10b61188652d42169fb6841a1beef057/users/71bda529f96349b3ba1d707695bfa74e/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.440587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b61188652d42169fb6841a1beef057, user_id=71bda529f96349b3ba1d707695bfa74e, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:26.443840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:26.530579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1097/2198] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:26 2022] PUT /identity/v3/projects/10b61188652d42169fb6841a1beef057/users/71bda529f96349b3ba1d707695bfa74e/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:26.535252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.545944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc5420-2f97-45b3-874d-3f8421e2e11d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.606281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01d52-9ad0-41b9-8838-5605363a818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.606931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01d52-9ad0-41b9-8838-5605363a818d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.607191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01d52-9ad0-41b9-8838-5605363a818d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.607464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01d52-9ad0-41b9-8838-5605363a818d admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.608275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a01d52-9ad0-41b9-8838-5605363a818d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:26.609860 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a01d52-9ad0-41b9-8838-5605363a818d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:26.611297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:26.621879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1102/2199] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:26.626166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.637214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e4992-9e00-42f7-9a64-801243ff98f7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.642232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.642925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.643214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.643474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] PATH_INFO: `/v3/projects/10b61188652d42169fb6841a1beef057/users/71bda529f96349b3ba1d707695bfa74e/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.646047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b61188652d42169fb6841a1beef057, user_id=71bda529f96349b3ba1d707695bfa74e, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:26.648787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:26.660208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1098/2200] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:26 2022] PUT /identity/v3/projects/10b61188652d42169fb6841a1beef057/users/71bda529f96349b3ba1d707695bfa74e/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:26.739709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.740010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.740249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:26.771471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] MFA Rules not processed for user `71bda529f96349b3ba1d707695bfa74e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:26.772695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:26.857766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1103/2201] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:26.869949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:26.871832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:26.920976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:26.933608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77bdb4-aaa6-42f5-94e9-625454ae0796 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:26.938022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:26.942662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:26.942961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:26.943245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:27.014491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:27.015933 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:27.025536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1099/2202] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:36:34.332260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:34.342281 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf79f1-a375-4371-bd2e-43ddfdcc3d11 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:34.345885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:34.346514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:34.346848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:34.347264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:34.404966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:34.406959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:34.415917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1104/2203] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:37.027207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=29 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:37.523661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:37.532571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdb3d9-31f5-4a07-8f47-c3bc094174e1 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:37.537371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10c448-089d-4f81-b146-7d6f8fc77e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:37.537985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10c448-089d-4f81-b146-7d6f8fc77e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:37.538263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10c448-089d-4f81-b146-7d6f8fc77e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:37.538561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10c448-089d-4f81-b146-7d6f8fc77e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:37.547878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10c448-089d-4f81-b146-7d6f8fc77e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:37.549409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10c448-089d-4f81-b146-7d6f8fc77e7d service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:37.560023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1100/2204] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:43.283184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:43.371426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58772291-3c90-4540-a312-8d9ef5b0e210 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:43.381864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:43.381864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:43.381864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:43.381864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] PATH_INFO: `/v3/users/7c72e514966a43848524c53e5f61680e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:43.382270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c72e514966a43848524c53e5f61680e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:43.382270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:43.420752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:43.586466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c72e514966a43848524c53e5f61680e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:43.626815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c72e514966a43848524c53e5f61680e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:43.627217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c72e514966a43848524c53e5f61680e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:43.640956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c72e514966a43848524c53e5f61680e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:43.648044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c72e514966a43848524c53e5f61680e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:43.654321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1105/2205] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:43 2022] DELETE /identity/v3/users/7c72e514966a43848524c53e5f61680e => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:43.659759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:43.660367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:43.660844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:43.684367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:43.685859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:43.750773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1101/2206] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:43.761364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:43.763081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:43.840768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2dc4d-2462-43f0-9cfd-36561db58952 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:43.844742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:43.845396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:43.845676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:43.845949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:43.856249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:43.857704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:43.869619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1106/2207] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:44.390178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:44.390178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:44.427117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d9673-3769-4131-a206-4be3b05f0cda None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:44.433979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:44.434806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:44.435089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:44.435382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] PATH_INFO: `/v3/projects/39add67fc96f48e4a59198ab7e815c21` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:44.436493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39add67fc96f48e4a59198ab7e815c21)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:44.438444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:44.547661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39add67fc96f48e4a59198ab7e815c21', 'request_id': 'req-d06f93b7-2b91-445b-9fa5-479be29ca0a6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:44.666885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39add67fc96f48e4a59198ab7e815c21', 'request_id': 'req-d06f93b7-2b91-445b-9fa5-479be29ca0a6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:44.677840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39add67fc96f48e4a59198ab7e815c21', 'request_id': 'req-d06f93b7-2b91-445b-9fa5-479be29ca0a6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:44.690500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] The token cache is being invalidate because project 39add67fc96f48e4a59198ab7e815c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:44.696110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:44.696110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d06f93b7-2b91-445b-9fa5-479be29ca0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:44.696110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1102/2208] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:44 2022] DELETE /identity/v3/projects/39add67fc96f48e4a59198ab7e815c21 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:44.697288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:44.708171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f46230-ca60-4c3b-97aa-ba0aec20b5d1 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:44.771872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:44.772530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:44.772822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:44.773086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] PATH_INFO: `/v3/users/b7a0b9c587e14232bb0ad476c4b390ae` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:44.774231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a0b9c587e14232bb0ad476c4b390ae)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:44.781097 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:44.909660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ae483-7eea-416e-8a00-f81a3f37170b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:44.909947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ae483-7eea-416e-8a00-f81a3f37170b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:44.910218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ae483-7eea-416e-8a00-f81a3f37170b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:44.911721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1103/2209] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:36:44.917306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:44.917597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:44.917876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:44.943628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:44.944887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:45.017689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:45.021492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a0b9c587e14232bb0ad476c4b390ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.038145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1104/2210] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:36:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:36:45.050990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:45.052617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:45.058331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:45.116914 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c8587-e7eb-478d-9974-23a239b2b832 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:45.119114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a0b9c587e14232bb0ad476c4b390ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.119992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a0b9c587e14232bb0ad476c4b390ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.120843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4641d8b-4d95-4a0b-8d6c-5821a7f01d26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:45.121563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4641d8b-4d95-4a0b-8d6c-5821a7f01d26 service glance] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.121762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4641d8b-4d95-4a0b-8d6c-5821a7f01d26 service glance] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.122006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4641d8b-4d95-4a0b-8d6c-5821a7f01d26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.136108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a0b9c587e14232bb0ad476c4b390ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.141196 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4641d8b-4d95-4a0b-8d6c-5821a7f01d26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:45.142654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4641d8b-4d95-4a0b-8d6c-5821a7f01d26 service glance] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:45.143422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d417f52-5e61-44c6-b9db-6a5e43ad08f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a0b9c587e14232bb0ad476c4b390ae'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.150284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1107/2211] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:44 2022] DELETE /identity/v3/users/b7a0b9c587e14232bb0ad476c4b390ae => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:45.158181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1105/2212] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:45.354674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f47af-2742-4396-ac33-c398a00f7aaf None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.355054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f47af-2742-4396-ac33-c398a00f7aaf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.355396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f47af-2742-4396-ac33-c398a00f7aaf None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.357191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1108/2213] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:36:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:36:45.363065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.363368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.363666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.400408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:45.401701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:45.465404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1106/2214] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:36:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:45.476746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f47af-2742-4396-ac33-c398a00f7aaf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:45.478704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5e8f47af-2742-4396-ac33-c398a00f7aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:45.493999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f47af-2742-4396-ac33-c398a00f7aaf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:45.497811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33264a62-e0e2-4cc5-b755-5fee69e5fe11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:45.498579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264a62-e0e2-4cc5-b755-5fee69e5fe11 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.498926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264a62-e0e2-4cc5-b755-5fee69e5fe11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.499275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33264a62-e0e2-4cc5-b755-5fee69e5fe11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.507629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33264a62-e0e2-4cc5-b755-5fee69e5fe11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:45.509059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33264a62-e0e2-4cc5-b755-5fee69e5fe11 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:45.517509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1109/2215] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:45.574216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:45.577768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:45.618390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237c60-329c-4b67-a39e-d0d5cafdea6d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.618862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237c60-329c-4b67-a39e-d0d5cafdea6d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.619207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7237c60-329c-4b67-a39e-d0d5cafdea6d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.620777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1110/2216] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 03 01:36:45.626232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.626658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.627476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.651361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:45.651361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:45.662431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa7704-be96-4106-a87d-ed02e24ddb4a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:45.667226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:45.667867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.668170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.668447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] PATH_INFO: `/v3/projects/1e5fa480a2364906a5bb0585ef20adf2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.669581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5fa480a2364906a5bb0585ef20adf2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:45.671603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:45.723493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1111/2217] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:36:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:45.734113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:45.744252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901954b-4dc5-464b-afae-a08ca3d333e9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:45.747905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:45.748512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.748791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.749060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.763053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:45.763053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:45.776140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1112/2218] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:36:45.778999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5fa480a2364906a5bb0585ef20adf2', 'request_id': 'req-018bcc6c-97bc-4636-9720-771fb29ffb67'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.878489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5fa480a2364906a5bb0585ef20adf2', 'request_id': 'req-018bcc6c-97bc-4636-9720-771fb29ffb67'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.880654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:45.887032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5fa480a2364906a5bb0585ef20adf2', 'request_id': 'req-018bcc6c-97bc-4636-9720-771fb29ffb67'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.900789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] The token cache is being invalidate because project 1e5fa480a2364906a5bb0585ef20adf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:45.901087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.901777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:45.903511 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1107/2219] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:45 2022] DELETE /identity/v3/projects/1e5fa480a2364906a5bb0585ef20adf2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:45.907594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:45.920241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada70d0-7589-470b-b1c4-add1dbe827c1 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:45.988453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:45.989106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:45.989376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:45.989649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] PATH_INFO: `/v3/users/71bda529f96349b3ba1d707695bfa74e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:45.990922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bda529f96349b3ba1d707695bfa74e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:45.993016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:46.124206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:46.148901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:46.150897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:46.179485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018bcc6c-97bc-4636-9720-771fb29ffb67 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:46.185558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:46.186208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:46.186516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:46.186788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] PATH_INFO: `/v3/users/feba07a3c1bd42dfa56780062ea7aae3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:46.187977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feba07a3c1bd42dfa56780062ea7aae3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:46.190147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:46.214082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bda529f96349b3ba1d707695bfa74e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.314923 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bda529f96349b3ba1d707695bfa74e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.316351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bda529f96349b3ba1d707695bfa74e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.341079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bda529f96349b3ba1d707695bfa74e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.364797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-54469b38-e16e-4313-b08e-f32ee44f5d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bda529f96349b3ba1d707695bfa74e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.374461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1113/2220] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:45 2022] DELETE /identity/v3/users/71bda529f96349b3ba1d707695bfa74e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:46.383081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feba07a3c1bd42dfa56780062ea7aae3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.383993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feba07a3c1bd42dfa56780062ea7aae3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.405495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feba07a3c1bd42dfa56780062ea7aae3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.418657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feba07a3c1bd42dfa56780062ea7aae3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.477824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feba07a3c1bd42dfa56780062ea7aae3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:46.485140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1108/2221] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:46 2022] DELETE /identity/v3/users/feba07a3c1bd42dfa56780062ea7aae3 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:46.489866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:46.490390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:46.490734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:46.511643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:46.513011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:46.591784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1114/2222] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:46.602115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:46.603796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:46.679523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75112072-3702-40a1-a9f8-f82b780204a0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:46.683764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:46.684574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:46.684852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:46.685330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:46.705262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:46.707810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:46.718793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1109/2223] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:47.007658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:47.009108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:47.023280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b5a95-5fa1-4e2a-96f0-27b61688671d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:47.033426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:47.033426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.033426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.033426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] PATH_INFO: `/v3/projects/10b61188652d42169fb6841a1beef057` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.034028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b61188652d42169fb6841a1beef057)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:47.035104 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:47.183296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b61188652d42169fb6841a1beef057', 'request_id': 'req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.241050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:47.245581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:47.263238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f337e-5cdb-4549-8b4d-d75f50b58baa service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:47.278190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b61188652d42169fb6841a1beef057', 'request_id': 'req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.287208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b61188652d42169fb6841a1beef057', 'request_id': 'req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.303351 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] The token cache is being invalidate because project 10b61188652d42169fb6841a1beef057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:47.303724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.304344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.305881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1115/2224] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:47 2022] DELETE /identity/v3/projects/10b61188652d42169fb6841a1beef057 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:47.350670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:47.352287 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.352440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.352613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] PATH_INFO: `/v3/projects/174605548d884b8c89e2739692609de2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.354110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=174605548d884b8c89e2739692609de2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:47.356212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:47.358039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:47.518113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:47.519666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:47.538119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bfc6d-9fb5-4177-9df3-fcbdbbb3c2a8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:47.544890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174605548d884b8c89e2739692609de2', 'request_id': 'req-1cd650ef-ec50-4507-80d7-14f8f9371ac5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.566494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:47.598440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:47.599137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.599497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.599966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] PATH_INFO: `/v3/users/f6c0c784e29444d395e9814f204fd26e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.601484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c0c784e29444d395e9814f204fd26e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:47.603629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:47.609818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174605548d884b8c89e2739692609de2', 'request_id': 'req-1cd650ef-ec50-4507-80d7-14f8f9371ac5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.617197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174605548d884b8c89e2739692609de2', 'request_id': 'req-1cd650ef-ec50-4507-80d7-14f8f9371ac5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.630599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] The token cache is being invalidate because project 174605548d884b8c89e2739692609de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:47.639750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.643940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1cd650ef-ec50-4507-80d7-14f8f9371ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.648227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1110/2225] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:47 2022] DELETE /identity/v3/projects/174605548d884b8c89e2739692609de2 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:36:47.653496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d611-5588-4001-8c74-b6167031295b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.653844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d611-5588-4001-8c74-b6167031295b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.654254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d611-5588-4001-8c74-b6167031295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.677735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1ae4d611-5588-4001-8c74-b6167031295b None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:47.679863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1ae4d611-5588-4001-8c74-b6167031295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:47.688775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1111/2226] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:47.696643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.696936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.697206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.717076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:47.718282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:47.727065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1112/2227] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:47.731619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:47.741074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ccfa61-0f61-4bfb-9158-f3c3585e6959 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:47.745917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:47.746553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.746830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.747099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.747752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:36:47.748052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:47.749644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:47.763318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1113/2228] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:36:47.765864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:47.776645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e97d6-3f12-4ea5-b019-5e8ce47a858d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:47.781336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:47.782197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.782516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.782790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.783454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:47.786009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:47.801131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c0c784e29444d395e9814f204fd26e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.897002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c0c784e29444d395e9814f204fd26e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.897585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c0c784e29444d395e9814f204fd26e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.906595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1114/2229] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:36:47 2022] POST /identity/v3/projects => generated 397 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:36:47.908208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:47.911781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c0c784e29444d395e9814f204fd26e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.918944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2be6eaa2-6674-4924-8aee-ded37f1933da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c0c784e29444d395e9814f204fd26e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:47.925203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1116/2230] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:47 2022] DELETE /identity/v3/users/f6c0c784e29444d395e9814f204fd26e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:47.930512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:47.930890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:47.931251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:47.952659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:47.953882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:47.971316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305c9e-62e8-4e3a-946d-700b726f8caf admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.011589 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1117/2231] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:36:48.022604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:48.024152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:48.032169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.032911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.033197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.033473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.034308 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.036543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.086913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb4d9f-638a-4693-bd39-e26b9a604e64 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.090634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.091247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.091526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.091795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.101315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.102965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.112549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1118/2232] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:36:48.210958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1115/2233] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:36:47 2022] POST /identity/v3/users => generated 299 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:48.215091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:48.216567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:48.229280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93f2c6-2a6b-4c04-8265-6fd3202223aa service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.234082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.234723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.235003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.235275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.235917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.237553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.248673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1119/2234] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:48.253259 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:48.263339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f972d-7268-4dd7-a066-3dc2e7ad5bcc admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.267996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235ef26-00cc-4f95-9a39-24a31c6ed619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.269194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235ef26-00cc-4f95-9a39-24a31c6ed619 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.269555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235ef26-00cc-4f95-9a39-24a31c6ed619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.269829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235ef26-00cc-4f95-9a39-24a31c6ed619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.270512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235ef26-00cc-4f95-9a39-24a31c6ed619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.272117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235ef26-00cc-4f95-9a39-24a31c6ed619 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.282691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1116/2235] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:48.287186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:48.297225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652bf68-e844-490a-8c4a-52eab5900cf5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.302811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.303143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.303483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.303823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] PATH_INFO: `/v3/projects/cba01b70c4474cdb92f6fe4e6585d159/users/5e71c336dabc4cb0857ec03add57632e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.331988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba01b70c4474cdb92f6fe4e6585d159, user_id=5e71c336dabc4cb0857ec03add57632e, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.334901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.410034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:48.434301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1120/2236] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:48 2022] PUT /identity/v3/projects/cba01b70c4474cdb92f6fe4e6585d159/users/5e71c336dabc4cb0857ec03add57632e/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:48.446299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.446640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.446878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.474378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:48.491306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] MFA Rules not processed for user `5e71c336dabc4cb0857ec03add57632e`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:48.492526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:48.555627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:48.560805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:48.590581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1117/2237] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:36:48.608299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:48.612925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:48.687931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610ce66-e349-4b71-9d80-ac3d773ed001 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.687931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.688448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.688534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.688822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] PATH_INFO: `/v3/projects/9fcf666e7f704bfd8817469bfd76a933` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.701706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c532cf1-9110-4caf-b586-f4329b6dd0d1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:48.704060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcf666e7f704bfd8817469bfd76a933)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.709342 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.709899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfaed12-d710-4b60-ae7b-6be4c2417f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:48.710553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfaed12-d710-4b60-ae7b-6be4c2417f7d service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.710825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfaed12-d710-4b60-ae7b-6be4c2417f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.711725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfaed12-d710-4b60-ae7b-6be4c2417f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.723114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfaed12-d710-4b60-ae7b-6be4c2417f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:48.724730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfaed12-d710-4b60-ae7b-6be4c2417f7d service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:48.740232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1118/2238] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:48 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:48.857290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcf666e7f704bfd8817469bfd76a933', 'request_id': 'req-ba5664a7-2e02-4167-a70a-272e2cb5d120'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:48.917749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcf666e7f704bfd8817469bfd76a933', 'request_id': 'req-ba5664a7-2e02-4167-a70a-272e2cb5d120'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:48.925981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcf666e7f704bfd8817469bfd76a933', 'request_id': 'req-ba5664a7-2e02-4167-a70a-272e2cb5d120'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:48.939008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] The token cache is being invalidate because project 9fcf666e7f704bfd8817469bfd76a933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:36:48.939319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:48.940120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba5664a7-2e02-4167-a70a-272e2cb5d120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:48.941956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1121/2239] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:48 2022] DELETE /identity/v3/projects/9fcf666e7f704bfd8817469bfd76a933 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:48.952533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:48.953094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:48.953373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:48.974938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:48.976171 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:49.030174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:49.048334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1119/2240] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:49.051633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.052302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.052581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.073524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:49.074786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:49.083403 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1122/2241] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:49.088125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.098677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336ff9a-2cf4-4ecb-958c-afb79ae7fb16 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.103240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.103837 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.105324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.105324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.105324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:36:49.105324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.107131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.119566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1120/2242] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:36:49.124015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.135019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d03f4-b329-4473-b978-cf9c26389b62 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.142109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.143016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.144044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.144402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.145354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.147329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.272487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1123/2243] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:36:49 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:49.274654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.284302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acd802-eeac-44a6-b61d-0c3a394417af admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.357255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.357868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.358149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.358451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.359088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.361003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.454756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1121/2244] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:36:49 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:36:49.455999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.467643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe3e32-fe9e-41b7-9a29-0aed73726538 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.474904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.475279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.475559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.475830 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.476488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.478114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.489289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1124/2245] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:49.493513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.503773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d72950-cd97-47dd-99b5-5506f87adcf8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.508453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb6ccb-cc92-4783-bbb6-7a9ed084cde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.509063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ccb-cc92-4783-bbb6-7a9ed084cde8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.509346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ccb-cc92-4783-bbb6-7a9ed084cde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.509613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb6ccb-cc92-4783-bbb6-7a9ed084cde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.510315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb6ccb-cc92-4783-bbb6-7a9ed084cde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.512075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb6ccb-cc92-4783-bbb6-7a9ed084cde8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.523166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1122/2246] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:36:49.528068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.557022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df207a-bee7-4d65-a6b5-8bc49f7cfc2f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.571583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.572394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.572775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.573148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] PATH_INFO: `/v3/projects/efb12fa605334996a0e25888c90a108a/users/69028bf5296541b6bfe9595d684c6567/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.575878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:49.593023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb12fa605334996a0e25888c90a108a, user_id=69028bf5296541b6bfe9595d684c6567, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.596049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.674715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1125/2247] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:36:49 2022] PUT /identity/v3/projects/efb12fa605334996a0e25888c90a108a/users/69028bf5296541b6bfe9595d684c6567/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:49.679778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bdb73-4b5e-410b-b35c-459d7800c360 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.680070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bdb73-4b5e-410b-b35c-459d7800c360 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.680339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bdb73-4b5e-410b-b35c-459d7800c360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.704114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b22bdb73-4b5e-410b-b35c-459d7800c360 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:49.705333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b22bdb73-4b5e-410b-b35c-459d7800c360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:49.772279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1123/2248] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:49.778278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:49.780085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:49.843033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08148fe2-92c9-4a6e-89f6-b465f189ca74 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:49.848068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:49.848676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:49.848964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:49.849244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:49.859355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:49.860827 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:49.871549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1126/2249] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:50.039966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94047d8d-1c80-4f43-a0b8-2c6f5aad3fb0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:50.040476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94047d8d-1c80-4f43-a0b8-2c6f5aad3fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:50.040911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94047d8d-1c80-4f43-a0b8-2c6f5aad3fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:50.061599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-94047d8d-1c80-4f43-a0b8-2c6f5aad3fb0 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:50.063220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-94047d8d-1c80-4f43-a0b8-2c6f5aad3fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:50.137936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1124/2250] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:36:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:50.163270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:50.163688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:50.249936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672169c-25f3-489f-8f3b-c2a5d365c629 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:50.255619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:50.256253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:50.256533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:50.256803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:50.264868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:50.266627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:50.275245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1127/2251] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:36:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 03 01:36:51.647303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8604bd-f7cb-4b45-9fbc-5d55c0082d41 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:51.648550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8604bd-f7cb-4b45-9fbc-5d55c0082d41 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:51.649070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8604bd-f7cb-4b45-9fbc-5d55c0082d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:51.684075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:51.696755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fb8604bd-f7cb-4b45-9fbc-5d55c0082d41 None None] MFA Rules not processed for user `69028bf5296541b6bfe9595d684c6567`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:51.698048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb8604bd-f7cb-4b45-9fbc-5d55c0082d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:51.797649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1125/2252] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:51.815484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:51.817091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:51.896105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d313c5b-0024-4bc9-ba76-15f3ea69ec78 service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:51.899796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:51.900407 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:51.900682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:51.901124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:51.914227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:51.915758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:51.926846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1128/2253] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:55.690930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc4c49-01f6-4287-b1e0-4dbaa555aab0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:55.691548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc4c49-01f6-4287-b1e0-4dbaa555aab0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:55.691660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc4c49-01f6-4287-b1e0-4dbaa555aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:55.720303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-38fc4c49-01f6-4287-b1e0-4dbaa555aab0 None None] MFA Rules not processed for user `69028bf5296541b6bfe9595d684c6567`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:55.720303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-38fc4c49-01f6-4287-b1e0-4dbaa555aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:55.723909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1126/2254] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:57.999778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:58.002800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:58.022014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18773f58-e173-433a-9e41-af34acf4bddd service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:58.038453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:58.039604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:58.040052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:58.040474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] PATH_INFO: `/v3/users/5e71c336dabc4cb0857ec03add57632e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:58.045229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e71c336dabc4cb0857ec03add57632e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:58.047397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:58.064156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:36:58.528476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e71c336dabc4cb0857ec03add57632e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:58.609189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e71c336dabc4cb0857ec03add57632e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:58.610133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e71c336dabc4cb0857ec03add57632e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:58.642616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e71c336dabc4cb0857ec03add57632e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:58.667379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e71c336dabc4cb0857ec03add57632e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:58.685868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1129/2255] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:36:57 2022] DELETE /identity/v3/users/5e71c336dabc4cb0857ec03add57632e => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:36:58.692116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:58.693464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:58.693944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:58.725990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:36:58.727581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:58.868845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1127/2256] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:36:58.879534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:58.881462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:58.981409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6fa666-c3f1-4c51-a2d0-5fa5e6d13485 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:58.991680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7666a56-21d7-48b4-8f0b-844ea74d739f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:58.992699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7666a56-21d7-48b4-8f0b-844ea74d739f service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:58.993284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7666a56-21d7-48b4-8f0b-844ea74d739f service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:58.993768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7666a56-21d7-48b4-8f0b-844ea74d739f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:59.004732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7666a56-21d7-48b4-8f0b-844ea74d739f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:59.007730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7666a56-21d7-48b4-8f0b-844ea74d739f service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:59.026169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1130/2257] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:36:59.759567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:36:59.761047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:36:59.774373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b78ce2-4e55-458a-b491-b42ec6ef8544 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:36:59.779819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:36:59.780485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:36:59.780764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:36:59.781034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] PATH_INFO: `/v3/projects/cba01b70c4474cdb92f6fe4e6585d159` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:36:59.782472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba01b70c4474cdb92f6fe4e6585d159)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:36:59.785383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:36:59.886573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba01b70c4474cdb92f6fe4e6585d159', 'request_id': 'req-26fbe11f-da1c-451a-ab64-3c3c69e32d89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:59.981266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba01b70c4474cdb92f6fe4e6585d159', 'request_id': 'req-26fbe11f-da1c-451a-ab64-3c3c69e32d89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:36:59.988025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba01b70c4474cdb92f6fe4e6585d159', 'request_id': 'req-26fbe11f-da1c-451a-ab64-3c3c69e32d89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:37:00.003342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] The token cache is being invalidate because project cba01b70c4474cdb92f6fe4e6585d159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:37:00.003909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:37:00.004955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-26fbe11f-da1c-451a-ab64-3c3c69e32d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:37:00.007014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1128/2258] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:36:59 2022] DELETE /identity/v3/projects/cba01b70c4474cdb92f6fe4e6585d159 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:37:00.012103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.013602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.013602 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.038787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:00.040012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:00.111708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1131/2259] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:00.117362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.117362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.117362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.135484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:00.136677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:00.144585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1129/2260] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:00.148971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.159408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d7480-df50-42a5-a6c9-fd1bd21bd005 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.163993 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.164584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.164862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.165132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.165793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:37:00.166074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.167754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.179740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1132/2261] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:37:00.183966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.193492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d109be-baea-4662-b48a-fbfc5c410c66 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.198087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.198737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.202026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.202026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.202026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.202026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.211131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:00.272935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1130/2262] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:37:00 2022] POST /identity/v3/projects => generated 379 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:37:00.276942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.287716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e991b-2df0-4b52-be66-fded6f7b6297 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.304834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:00.344007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.344614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.344904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.345441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.347354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.349369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.420567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1133/2263] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:37:00 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:37:00.424647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.435320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89373-b7e9-4b65-860c-8d1abc19a45a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.440065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.440660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.440939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.441225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.441927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.443564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.453489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1131/2264] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:37:00.458117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.468343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce385e-bfba-4b81-91db-97bbc06c7d36 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.473707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e7992-4257-432a-a257-a8d88251b5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.474304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e7992-4257-432a-a257-a8d88251b5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.474688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e7992-4257-432a-a257-a8d88251b5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.474898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e7992-4257-432a-a257-a8d88251b5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.475513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e7992-4257-432a-a257-a8d88251b5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.477133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e7992-4257-432a-a257-a8d88251b5c8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.489220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1134/2265] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:37:00.493576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.504577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f09c5-0d9e-40ab-ba9a-ccb34fd8aed6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.511932 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.513066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.513568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.514078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] PATH_INFO: `/v3/projects/c306d09e93584320b34697dba7641ac8/users/df0ffa9f450642cc94ed828cb34034b5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.532009 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c306d09e93584320b34697dba7641ac8, user_id=df0ffa9f450642cc94ed828cb34034b5, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.535508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.574757 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1132/2266] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:37:00 2022] PUT /identity/v3/projects/c306d09e93584320b34697dba7641ac8/users/df0ffa9f450642cc94ed828cb34034b5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:37:00.579789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.580077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.581003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.600304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:00.601544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:00.660437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1135/2267] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:00.670029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:00.671752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:00.728993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768a654-5834-419a-b9a1-a43e9782a7e8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:00.732858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3091b2b-96fd-451b-9b7b-5979d3861e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:00.733471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091b2b-96fd-451b-9b7b-5979d3861e0e service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:00.733752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091b2b-96fd-451b-9b7b-5979d3861e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:00.734020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091b2b-96fd-451b-9b7b-5979d3861e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:00.744613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3091b2b-96fd-451b-9b7b-5979d3861e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:00.747648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3091b2b-96fd-451b-9b7b-5979d3861e0e service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:00.761792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1133/2268] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:37:05.037356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:05.055986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495bbfbe-b16c-4b0c-a3d6-f62544f26f62 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:05.060617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:05.061590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:05.062106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:05.062617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:05.064805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:05.159861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:05.161494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e9a11-38be-46d5-a52f-6932c135f8e3 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:05.172320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1136/2269] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:37:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:37:10.401288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:10.402088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:10.402088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:10.439893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] MFA Rules not processed for user `409c324dca974d9dad9529b2a0d82b30`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:10.442003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:10.525791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1134/2270] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:11.094190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:11.095242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:11.095807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:11.199266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] MFA Rules not processed for user `df0ffa9f450642cc94ed828cb34034b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:11.202442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:11.270396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:11.356769 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1137/2271] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:11.361662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:11.376118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff7732-ec29-45ac-a32b-9b1ac138a36b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:11.385088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:11.386053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:11.386571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:11.386990 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:11.388347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:11.391639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:11.515917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1135/2272] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:37:11 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:37:11.520456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:11.534982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3592eca-5c96-4b56-8757-09b6b2d2dde1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:11.640529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:11.646931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:11.650102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:11.650604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:11.651504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:11.653648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:11.772170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1138/2273] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:37:11 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:37:11.775812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:11.792533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7ff5-3bfa-4757-bd35-d986793e1a4d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:11.812362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:11.813227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:11.813862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:11.818693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:11.819900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:11.827491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:11.841657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1136/2274] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:37:11.845718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:11.856109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac8fba-219c-4a67-a924-47d7630d2be5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:11.866435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:11.872244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:11.872244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:11.872244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] PATH_INFO: `/v3/projects/7bf08f2124de40729635b1c5d31b1dcd/users/269c8434b1c64bd38729ac25ac1eaf48/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:11.904251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf08f2124de40729635b1c5d31b1dcd, user_id=269c8434b1c64bd38729ac25ac1eaf48, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:11.915703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:12.039558 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1139/2275] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:37:11 2022] PUT /identity/v3/projects/7bf08f2124de40729635b1c5d31b1dcd/users/269c8434b1c64bd38729ac25ac1eaf48/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:37:12.045675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:12.058978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ce1e3-1e6f-4159-9494-2cb619dad505 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:12.109869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:12.160059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb36661-33cd-47ad-98e2-0228a8492ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:12.160870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36661-33cd-47ad-98e2-0228a8492ebd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:12.161297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36661-33cd-47ad-98e2-0228a8492ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:12.161731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb36661-33cd-47ad-98e2-0228a8492ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:12.162678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36661-33cd-47ad-98e2-0228a8492ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:12.184948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb36661-33cd-47ad-98e2-0228a8492ebd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:12.190697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1137/2276] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:37:12.193607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:12.208770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3501-93d2-474b-b1e5-9efbd54c713b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:12.221080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:12.221771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:12.222088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:12.222380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] PATH_INFO: `/v3/projects/7bf08f2124de40729635b1c5d31b1dcd/users/269c8434b1c64bd38729ac25ac1eaf48/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:12.224458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf08f2124de40729635b1c5d31b1dcd, user_id=269c8434b1c64bd38729ac25ac1eaf48, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:12.227243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:12.249731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1140/2277] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:37:12 2022] PUT /identity/v3/projects/7bf08f2124de40729635b1c5d31b1dcd/users/269c8434b1c64bd38729ac25ac1eaf48/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:37:18.748883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43a907-f252-4667-bd71-d70f1623cc60 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:18.749431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43a907-f252-4667-bd71-d70f1623cc60 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:18.749516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43a907-f252-4667-bd71-d70f1623cc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:18.801902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ae43a907-f252-4667-bd71-d70f1623cc60 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:18.806796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ae43a907-f252-4667-bd71-d70f1623cc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:18.963450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1138/2278] 10.222.0.13 () {58 vars in 967 bytes} [Sat Sep 3 01:37:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:37:18.992372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:18.995617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:19.127689 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2dd21-41bf-4c05-a8ca-0d3edda874b8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:19.134573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:19.135840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:19.135840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:19.136188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:19.151397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:19.155447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:19.169109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1141/2279] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:37:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:37:20.684979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d565c5-0a2f-4d06-93f3-098e7903d4ed None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:20.685555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d565c5-0a2f-4d06-93f3-098e7903d4ed None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:20.685555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d565c5-0a2f-4d06-93f3-098e7903d4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:20.729502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f8d565c5-0a2f-4d06-93f3-098e7903d4ed None None] MFA Rules not processed for user `269c8434b1c64bd38729ac25ac1eaf48`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:20.730790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f8d565c5-0a2f-4d06-93f3-098e7903d4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:20.910777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1139/2280] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:20.918612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:20.922058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:20.936080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:21.044956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dd811-d8c9-4464-bb79-9d9596398281 service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:21.051127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3176ed5-2aa6-438c-b5d6-002abc42843f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:21.051990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176ed5-2aa6-438c-b5d6-002abc42843f service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:21.052277 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176ed5-2aa6-438c-b5d6-002abc42843f service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:21.052553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176ed5-2aa6-438c-b5d6-002abc42843f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:21.116085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3176ed5-2aa6-438c-b5d6-002abc42843f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:21.117705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3176ed5-2aa6-438c-b5d6-002abc42843f service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:21.128298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1142/2281] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:37:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:37:29.971859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:29.972349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:29.972506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:29.999882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] MFA Rules not processed for user `2b9ab1983bb84318b509878b930e9402`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:30.001911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:30.067208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:30.078565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1140/2282] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:37:31.147171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=54 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:37:51.213803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:51.214446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:51.214446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:51.233341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] MFA Rules not processed for user `df0ffa9f450642cc94ed828cb34034b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:37:51.234569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:51.242080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1143/2283] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:37:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:37:52.284742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:52.290000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:52.355270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc656b26-d3dc-4052-afd6-2ff6f15afe53 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:52.359042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:52.359664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:52.360285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:52.360556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:52.362470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:52.380829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:52.382706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:52.395459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1141/2284] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:37:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:37:55.771313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:37:55.795542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62daa85a-e318-45d4-b319-07509f2ed475 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:37:55.815340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:37:55.816264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:37:55.816717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:37:55.817222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:37:55.821627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:37:55.850098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:37:55.851597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:37:55.878907 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1144/2285] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:37:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:02.390149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=26 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:05.873347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=9 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:09.882481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:09.883970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:09.897891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939932f-4a76-45f3-8ae5-04927887450f service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:09.903471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:09.904072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:09.904546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:09.904671 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:09.906330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:09.969068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:09.970690 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:09.980503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1142/2286] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:10.576574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:10.588371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc18d6a-396d-4474-aedd-c903dd8efce4 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:10.593988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:10.594622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:10.594906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:10.595181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:10.596917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:10.655038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:10.656650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:10.666667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1145/2287] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:19.976267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=28 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:20.661813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=28 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:23.783539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:23.796953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc2e9b-d847-474e-b909-1040a2a8f885 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:23.800637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:23.801240 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:23.801518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:23.802093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:23.812331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:23.813769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:23.823806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1143/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:28.349969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:28.350608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:28.442762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39ec20-3271-44cd-9238-832bbe3b2f6b service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:28.446868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:28.447839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:28.448228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:28.448603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:28.450688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:28.531568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:28.533395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf44ed-e59d-42d0-8a14-da2fe04fd2d8 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:28.548708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1146/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:29.161292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:29.170463 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e58a99-f052-4704-8357-832aed49163d service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:29.174077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:29.174724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:29.175002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:29.175275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:29.177008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:29.188315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:29.190687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:29.199237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1144/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:32.272940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe81c0-31de-4333-8649-2c437b125202 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:32.273446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe81c0-31de-4333-8649-2c437b125202 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:32.273573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe81c0-31de-4333-8649-2c437b125202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:32.293616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-babe81c0-31de-4333-8649-2c437b125202 None None] MFA Rules not processed for user `29b4f0836bbf4684bdc6177e42b88b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:32.294870 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-babe81c0-31de-4333-8649-2c437b125202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:32.301717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1147/2291] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:32.632290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:32.640801 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ce38e-3ead-413d-a095-4ea955ae749d service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:32.644423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:32.645013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:32.645292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:32.645564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:32.647390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:32.656634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:32.658316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:32.666556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1145/2292] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:33.328156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe81c0-31de-4333-8649-2c437b125202 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:33.329953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-babe81c0-31de-4333-8649-2c437b125202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:33.467355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe81c0-31de-4333-8649-2c437b125202 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:33.472943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:33.473568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:33.473845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:33.474115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:33.476353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:33.488900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:33.491142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6e778-fbdf-43b4-a0f3-662c2eeba524 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:33.502197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1148/2293] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:38:34.739767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:34.757531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08e345-def8-45f9-a6b1-fa9478c37adc service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:34.764491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:34.765117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:34.765396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:34.765858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:34.769220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:34.783372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:34.785022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:34.796908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1146/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:39.809983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8b0b7-2631-48fb-80db-a1b32586f45c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:39.810773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8b0b7-2631-48fb-80db-a1b32586f45c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:39.811206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8b0b7-2631-48fb-80db-a1b32586f45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:39.834961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9cf8b0b7-2631-48fb-80db-a1b32586f45c None None] MFA Rules not processed for user `69028bf5296541b6bfe9595d684c6567`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:39.836183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9cf8b0b7-2631-48fb-80db-a1b32586f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:39.880176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:39.906665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1149/2295] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:40.174862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:40.185241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150ed31-eabb-4006-9588-8c96bfc20a3b service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:40.189154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:40.189771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:40.190047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:40.190316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:40.192060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:40.213971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:40.215589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:40.226424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1147/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:44.221883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:44.221883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:44.221883 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:44.240698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] MFA Rules not processed for user `ecf076dde874415aa734644f27a10b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:44.241912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:44.249599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1150/2297] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:48.138760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:48.159083 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb1a05-0fa6-4038-bc24-be58e7cdc8fa service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:48.164976 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:48.165913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:48.166262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:48.166658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:48.168684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:48.183265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:48.185098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:48.199620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1148/2298] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:49.344812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:49.355157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a3c03-cd75-4a71-b976-e4a42ca45654 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:49.358743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:49.359353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:49.359637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:49.359916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:49.361683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:49.441575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:49.443127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:49.455044 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1151/2299] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:50.118194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:50.125769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=37 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:50.132780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89c559-4652-41a3-9a53-b0aee86caf45 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:50.138359 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:50.138986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:50.139346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:50.139621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:50.141377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:50.154835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:50.156621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:50.167335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1149/2300] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:54.472661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:54.476238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:54.504361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472d971-786c-4aa5-a01d-f44e977cbe29 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:54.506957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:54.508269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:54.508971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:54.509380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] PATH_INFO: `/v3/users/409c324dca974d9dad9529b2a0d82b30` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:54.510909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=409c324dca974d9dad9529b2a0d82b30)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:54.513667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:54.655006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:54.670031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac56a33-f11c-45c9-9c27-e0facb52a051 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:54.676443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:54.682059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:54.682059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:54.682059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:54.682059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:54.694841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:54.703170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:54.752297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1150/2301] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:38:54.778047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409c324dca974d9dad9529b2a0d82b30'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:54.878374 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409c324dca974d9dad9529b2a0d82b30'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:54.880597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409c324dca974d9dad9529b2a0d82b30'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:54.899607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409c324dca974d9dad9529b2a0d82b30'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:54.909130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-144ebfa1-66ed-4a6d-8687-792614ae4589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409c324dca974d9dad9529b2a0d82b30'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:54.923267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1152/2302] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:38:54 2022] DELETE /identity/v3/users/409c324dca974d9dad9529b2a0d82b30 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:38:55.284015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:55.293731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b6067-1e10-4213-ac42-769a91cb75a8 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:55.388395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:55.389241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:55.389661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:55.390255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] PATH_INFO: `/v3/projects/d93c5a3ef20c4a5b87af4803f820b60e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:55.398539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93c5a3ef20c4a5b87af4803f820b60e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:55.398539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:55.518010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93c5a3ef20c4a5b87af4803f820b60e', 'request_id': 'req-937e3d65-314a-434f-8d40-b0c43e21b9a1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:55.590784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93c5a3ef20c4a5b87af4803f820b60e', 'request_id': 'req-937e3d65-314a-434f-8d40-b0c43e21b9a1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:55.602763 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93c5a3ef20c4a5b87af4803f820b60e', 'request_id': 'req-937e3d65-314a-434f-8d40-b0c43e21b9a1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:55.621290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] The token cache is being invalidate because project d93c5a3ef20c4a5b87af4803f820b60e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:38:55.624870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:55.624870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-937e3d65-314a-434f-8d40-b0c43e21b9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:38:55.624870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1151/2303] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:38:55 2022] DELETE /identity/v3/projects/d93c5a3ef20c4a5b87af4803f820b60e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:38:55.629676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:55.630897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:55.631393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:55.654484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:55.656123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:55.699799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:55.721471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1153/2304] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:55.729666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:55.730175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:55.730175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:55.751172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:55.752646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:55.762537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1152/2305] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:55.769128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:55.780843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12345e1-3bbf-4b60-ae51-f7b6786b0091 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:55.788763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:55.789387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:55.789713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:55.790049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:55.790732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:38:55.791093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:55.792746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:55.810630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1154/2306] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:38:55.813969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:55.824509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b98a8-d59e-439f-adb0-0c3efdef1f8f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:55.829640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92320529-e725-490d-a0c4-373876814579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:55.830611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92320529-e725-490d-a0c4-373876814579 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:55.831281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92320529-e725-490d-a0c4-373876814579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:55.832154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92320529-e725-490d-a0c4-373876814579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:55.832883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92320529-e725-490d-a0c4-373876814579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:55.834834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92320529-e725-490d-a0c4-373876814579 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:55.933572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1153/2307] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:38:55 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:38:55.934777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:55.945842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f768b-47d8-4c3a-bd44-a87311c1e906 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:56.010576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:56.011353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:56.011665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:56.011901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:56.012547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:56.014571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:56.268286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1155/2308] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:38:55 2022] POST /identity/v3/users => generated 282 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:38:56.269682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92320529-e725-490d-a0c4-373876814579 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:56.281569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92320529-e725-490d-a0c4-373876814579 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:56.287542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:56.288153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:56.288433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:56.288716 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:56.289500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:56.291245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:56.301689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1154/2309] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:38:56.306574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:56.318147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb650e-4c84-47b5-8231-782113ff227e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:56.322896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1e4e4-d31d-4402-aaf9-c7dd7c0b43d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:56.323503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e4e4-d31d-4402-aaf9-c7dd7c0b43d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:56.323787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e4e4-d31d-4402-aaf9-c7dd7c0b43d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:56.324058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e4e4-d31d-4402-aaf9-c7dd7c0b43d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:56.324700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1e4e4-d31d-4402-aaf9-c7dd7c0b43d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:56.326571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1e4e4-d31d-4402-aaf9-c7dd7c0b43d7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:56.337905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1156/2310] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:38:56.342201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:56.353398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dd44e-9223-40ad-894c-efabda51ddb9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:56.358577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:56.359236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:56.359522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:56.359796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] PATH_INFO: `/v3/projects/6788366267634586abdb33471f05d178/users/18b6a9793ee34b5684967c5f97d4ecf7/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:56.377070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6788366267634586abdb33471f05d178, user_id=18b6a9793ee34b5684967c5f97d4ecf7, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:56.379931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:56.478838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1155/2311] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:38:56 2022] PUT /identity/v3/projects/6788366267634586abdb33471f05d178/users/18b6a9793ee34b5684967c5f97d4ecf7/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:38:56.492346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:56.492635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:56.498168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:56.542268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] MFA Rules not processed for user `18b6a9793ee34b5684967c5f97d4ecf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:56.545771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:56.634032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1157/2312] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:56.638172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:56.646369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:56.800097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f866b34-1854-4092-9f63-aef18897bd46 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:56.832759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:56.833979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:56.836755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:56.837072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:56.837927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:56.845041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:56.974660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1156/2313] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:38:56 2022] POST /identity/v3/projects => generated 365 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:38:56.978831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:56.994574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93658f7a-eb2b-40c1-aeeb-0bc2a1dd441d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.040785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:57.070216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.071215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.071809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.072238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.073533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:57.075720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:57.188765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1158/2314] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:38:56 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:38:57.198981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:57.211211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450999-2509-49aa-8858-354be0e6b5c1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.216224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.216850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.217135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.217496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.218073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:57.219759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:57.234487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1157/2315] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:38:57.235746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:57.248863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7a000-5a11-428b-a179-fb600b39d6f0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.254640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.256088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.256440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.256793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] PATH_INFO: `/v3/projects/293be8bfb24b45ec897eda71741f3ad9/users/c7e7f9cec8ac47d6b1e50fab81a09e8c/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.283325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293be8bfb24b45ec897eda71741f3ad9, user_id=c7e7f9cec8ac47d6b1e50fab81a09e8c, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:57.287153 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:57.368915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1159/2316] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:38:57 2022] PUT /identity/v3/projects/293be8bfb24b45ec897eda71741f3ad9/users/c7e7f9cec8ac47d6b1e50fab81a09e8c/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:38:57.373180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:57.384705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bdb6-3094-4dd2-a4b3-310fadde8e3a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.449856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cc79d-cc17-424f-a6a4-7eddb7d3b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.450503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cc79d-cc17-424f-a6a4-7eddb7d3b082 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.450841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cc79d-cc17-424f-a6a4-7eddb7d3b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.451271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cc79d-cc17-424f-a6a4-7eddb7d3b082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.451972 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cc79d-cc17-424f-a6a4-7eddb7d3b082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:57.453653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cc79d-cc17-424f-a6a4-7eddb7d3b082 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:57.470165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1158/2317] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:38:57.474039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:57.485273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4e958-75c4-4949-bf36-23abcb52add0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.490902 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.491620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.491934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.492208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] PATH_INFO: `/v3/projects/293be8bfb24b45ec897eda71741f3ad9/users/c7e7f9cec8ac47d6b1e50fab81a09e8c/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.494162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293be8bfb24b45ec897eda71741f3ad9, user_id=c7e7f9cec8ac47d6b1e50fab81a09e8c, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:57.496881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:57.508053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1160/2318] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:38:57 2022] PUT /identity/v3/projects/293be8bfb24b45ec897eda71741f3ad9/users/c7e7f9cec8ac47d6b1e50fab81a09e8c/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:38:57.518372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.518668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.518936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.548891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] MFA Rules not processed for user `c7e7f9cec8ac47d6b1e50fab81a09e8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:38:57.550142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:57.618445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:38:57.630550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1159/2319] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:38:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:38:57.637381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:57.639597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:38:57.721820 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa6550-da43-4752-a1da-607833791775 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.726833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.727459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.727739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.728054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:57.737754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:57.739545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:57.749685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1161/2320] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:38:57.961944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:38:57.973735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512bc80-f981-499d-adf2-6a9eb06a8954 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:38:57.982322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb598d9b-fb3e-4410-8308-681b405d8f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:38:57.983325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb598d9b-fb3e-4410-8308-681b405d8f0f service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:38:57.983790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb598d9b-fb3e-4410-8308-681b405d8f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:38:57.984303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb598d9b-fb3e-4410-8308-681b405d8f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:38:58.072657 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb598d9b-fb3e-4410-8308-681b405d8f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:38:58.076053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb598d9b-fb3e-4410-8308-681b405d8f0f service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:38:58.092184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1160/2321] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:38:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:01.698003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:01.776064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207697b0-13af-4991-924a-c410c4559212 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:01.781268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:01.785188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:01.785188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:01.785188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] PATH_INFO: `/v3/users/18b6a9793ee34b5684967c5f97d4ecf7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:01.785188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b6a9793ee34b5684967c5f97d4ecf7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:01.787466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:01.964804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:01.986577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b6a9793ee34b5684967c5f97d4ecf7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:02.070762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b6a9793ee34b5684967c5f97d4ecf7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:02.071434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b6a9793ee34b5684967c5f97d4ecf7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:02.104500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b6a9793ee34b5684967c5f97d4ecf7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:02.114231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b6a9793ee34b5684967c5f97d4ecf7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:02.124973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1162/2322] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:01 2022] DELETE /identity/v3/users/18b6a9793ee34b5684967c5f97d4ecf7 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:02.129338 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:02.130425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:02.130557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:02.153966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:02.155251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:02.271609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1161/2323] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:39:02.290683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:02.293179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:02.373290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978dfb7-c548-48ab-885f-1fa0f8c063c6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:02.378999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:02.380108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:02.380510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:02.380924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:02.397839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:02.399796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:02.419461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1163/2324] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:02.958767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:02.960269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:02.975156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884bcc0-d5d2-4006-aec7-25aae6812fa8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:02.982044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:02.982766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:02.983072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:02.983322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] PATH_INFO: `/v3/projects/6788366267634586abdb33471f05d178` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:02.992807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6788366267634586abdb33471f05d178)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:02.994980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:03.117997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6788366267634586abdb33471f05d178', 'request_id': 'req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.356652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6788366267634586abdb33471f05d178', 'request_id': 'req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.362742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6788366267634586abdb33471f05d178', 'request_id': 'req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.373938 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] The token cache is being invalidate because project 6788366267634586abdb33471f05d178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:03.374170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.374820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.376301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1162/2325] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:02 2022] DELETE /identity/v3/projects/6788366267634586abdb33471f05d178 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:03.380786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:03.390529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cb853-1ab7-4e28-9c75-81695acf4518 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:03.449971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:03.450636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:03.450916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:03.451185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] PATH_INFO: `/v3/users/c7e7f9cec8ac47d6b1e50fab81a09e8c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:03.452329 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e7f9cec8ac47d6b1e50fab81a09e8c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:03.454280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:03.661041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e7f9cec8ac47d6b1e50fab81a09e8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.756185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e7f9cec8ac47d6b1e50fab81a09e8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.756873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e7f9cec8ac47d6b1e50fab81a09e8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.759084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:03.769383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e7f9cec8ac47d6b1e50fab81a09e8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.776608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb956495-2e90-4f8d-9972-5267a6d16803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e7f9cec8ac47d6b1e50fab81a09e8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:03.782548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1164/2326] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:03 2022] DELETE /identity/v3/users/c7e7f9cec8ac47d6b1e50fab81a09e8c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:04.243785 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:04.245370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:04.261074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e451f-3f59-4c19-8d3d-57e579d2dbd4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:04.278834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:04.319425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:04.320062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:04.320332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:04.320605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] PATH_INFO: `/v3/projects/293be8bfb24b45ec897eda71741f3ad9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:04.321714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293be8bfb24b45ec897eda71741f3ad9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:04.323655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:04.470842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293be8bfb24b45ec897eda71741f3ad9', 'request_id': 'req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:04.544414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293be8bfb24b45ec897eda71741f3ad9', 'request_id': 'req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:04.550354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293be8bfb24b45ec897eda71741f3ad9', 'request_id': 'req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:04.562496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] The token cache is being invalidate because project 293be8bfb24b45ec897eda71741f3ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:04.571095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:04.571095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbbf0c0e-d0d0-46a7-b218-2d3e8fd49e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:04.571095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1163/2327] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:04 2022] DELETE /identity/v3/projects/293be8bfb24b45ec897eda71741f3ad9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:04.571427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:04.571427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:04.571427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:04.595112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:04.597178 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:04.665930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1165/2328] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:04.673877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:04.674163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:04.674474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:04.700580 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:04.701986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:04.712327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1164/2329] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:04.716163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:04.726742 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e521-e141-4a83-86a9-ba16f03208e4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:04.731275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:04.731875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:04.732160 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:04.732427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:04.733174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:39:04.733481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:04.735135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:04.747050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1166/2330] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:39:04.751388 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:04.760796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b206115-a109-4fca-a6db-2b700e7cf9ee None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:04.765426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:04.766010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:04.766285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:04.766588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:04.767216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:04.769092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:04.855496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1165/2331] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:39:04 2022] POST /identity/v3/projects => generated 399 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:39:04.859864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:04.870174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54638e-96ff-4702-b503-9ffca028ee66 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:04.927081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:04.927641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:04.927951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:04.928229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:04.928865 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:04.930852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.025349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1167/2332] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:39:04 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:05.029561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.041178 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d362dc-d619-474c-8437-88488e335b7d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.045800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.046423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.048152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.048152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.048152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.049189 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.059788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1166/2333] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:05.063715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.080688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9dc61-e981-4a60-af51-5dd553cc139d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.080688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c82e7d-b96e-4edc-aa4c-853c0a232608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.080688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c82e7d-b96e-4edc-aa4c-853c0a232608 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.081211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c82e7d-b96e-4edc-aa4c-853c0a232608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.081515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c82e7d-b96e-4edc-aa4c-853c0a232608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.082139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c82e7d-b96e-4edc-aa4c-853c0a232608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.084417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c82e7d-b96e-4edc-aa4c-853c0a232608 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.095425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1168/2334] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:05.099731 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.110187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52c27d-1910-49b4-8ebb-3ef35884ef46 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.115114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.115769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.116048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.116316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] PATH_INFO: `/v3/projects/378e3decbdca4e779c2b488b2a1d738f/users/bf5a1a990bfe4942a54c4a6d4698b5ce/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.131402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=378e3decbdca4e779c2b488b2a1d738f, user_id=bf5a1a990bfe4942a54c4a6d4698b5ce, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.134092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.201841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1167/2335] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:05 2022] PUT /identity/v3/projects/378e3decbdca4e779c2b488b2a1d738f/users/bf5a1a990bfe4942a54c4a6d4698b5ce/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:05.214102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.214408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.214685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.244803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] MFA Rules not processed for user `bf5a1a990bfe4942a54c4a6d4698b5ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:05.246228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:05.289028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:05.327188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1169/2336] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:05.331744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.380924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:05.397203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd9169-172c-40ec-bdfd-2aad45ee4162 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.403423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.404084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.404302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.404574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.405211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.414686 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.483727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1168/2337] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:39:05 2022] POST /identity/v3/projects => generated 399 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:05.487869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.500113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb7d12-450a-415b-a29e-7b7aab771ab9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.558615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.559142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.559421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.559687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.560309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.562286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.649228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1170/2338] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:39:05 2022] POST /identity/v3/users => generated 306 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:39:05.653285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.663819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c69c2a-3691-4729-97b7-a901e92dba31 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.669255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.669859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.670134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.670428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.672889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.672889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.683979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1169/2339] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:05.688301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.699257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73cc46-3ddf-4b42-bbfa-bda905d43364 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.704076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.704733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.705007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.705268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] PATH_INFO: `/v3/projects/b48af8a4bac64df2bab596137c7be7f6/users/1cd109ac62ad4768b906d76c33ffd021/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.721113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48af8a4bac64df2bab596137c7be7f6, user_id=1cd109ac62ad4768b906d76c33ffd021, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.727424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.833713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1171/2340] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:05 2022] PUT /identity/v3/projects/b48af8a4bac64df2bab596137c7be7f6/users/1cd109ac62ad4768b906d76c33ffd021/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:05.835039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.845700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8718bc3-75b3-4531-acdc-b9b608b20f5f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.903224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792d862-b617-4400-bce7-b09840463ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.903835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792d862-b617-4400-bce7-b09840463ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.904113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792d862-b617-4400-bce7-b09840463ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.904382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792d862-b617-4400-bce7-b09840463ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.905080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792d862-b617-4400-bce7-b09840463ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.906741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792d862-b617-4400-bce7-b09840463ba2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.917638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1170/2341] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:05.921803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:05.933414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf16fa-8d6a-47e5-8039-0a79af2a4f60 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:05.938068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:05.938759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.939038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.939309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] PATH_INFO: `/v3/projects/b48af8a4bac64df2bab596137c7be7f6/users/1cd109ac62ad4768b906d76c33ffd021/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:05.941224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48af8a4bac64df2bab596137c7be7f6, user_id=1cd109ac62ad4768b906d76c33ffd021, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:05.944150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:05.954555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1172/2342] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:05 2022] PUT /identity/v3/projects/b48af8a4bac64df2bab596137c7be7f6/users/1cd109ac62ad4768b906d76c33ffd021/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:39:05.971047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:05.971272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:05.971544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:06.000208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] MFA Rules not processed for user `1cd109ac62ad4768b906d76c33ffd021`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:06.001456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:06.070878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1171/2343] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:06.081613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:06.083378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:06.143027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e81e5d-80d0-4b89-9ac9-1cb4667bf316 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:06.147604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:06.148258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:06.148648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:06.148925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:06.158695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:06.160135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:06.170058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1173/2344] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:06.394603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:06.403949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89cf6-aaa8-4743-b692-82a0c6194384 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:06.407708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460b0b-5c41-4bf6-ad69-b57d66ac2f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:06.410092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460b0b-5c41-4bf6-ad69-b57d66ac2f2a service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:06.410092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460b0b-5c41-4bf6-ad69-b57d66ac2f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:06.410092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460b0b-5c41-4bf6-ad69-b57d66ac2f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:06.452480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:06.471892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460b0b-5c41-4bf6-ad69-b57d66ac2f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:06.474351 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460b0b-5c41-4bf6-ad69-b57d66ac2f2a service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:06.482442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1172/2345] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:11.575868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:11.604451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:11.639462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e283e-365e-49dc-a2e2-d5fbded3a35f service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:11.644494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:11.645119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:11.645395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:11.645668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] PATH_INFO: `/v3/users/bf5a1a990bfe4942a54c4a6d4698b5ce` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:11.647300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5a1a990bfe4942a54c4a6d4698b5ce)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:11.649267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:11.860296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5a1a990bfe4942a54c4a6d4698b5ce'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:11.927657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5a1a990bfe4942a54c4a6d4698b5ce'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:11.928311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5a1a990bfe4942a54c4a6d4698b5ce'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:11.941139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5a1a990bfe4942a54c4a6d4698b5ce'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:11.948726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5a1a990bfe4942a54c4a6d4698b5ce'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:11.954759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1174/2346] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:11 2022] DELETE /identity/v3/users/bf5a1a990bfe4942a54c4a6d4698b5ce => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:11.959881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:11.960441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:11.960441 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:11.979229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:11.980428 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:12.040977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1173/2347] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:12.053224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:12.055070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:12.112597 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbeeb1-1559-4aac-9cf6-0e50d1eec886 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:12.116790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:12.117406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:12.117687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:12.117957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:12.129441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:12.131560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:12.143333 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1175/2348] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:12.535197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:12.536979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:12.553674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d064e-d967-437c-9f3c-4d1cf18e8bcc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:12.560491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:12.560639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:12.560873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:12.561142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] PATH_INFO: `/v3/projects/378e3decbdca4e779c2b488b2a1d738f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:12.565258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=378e3decbdca4e779c2b488b2a1d738f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:12.565258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:12.694699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '378e3decbdca4e779c2b488b2a1d738f', 'request_id': 'req-f5ce48fc-651b-4b57-ac60-988040706c55'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:12.773643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '378e3decbdca4e779c2b488b2a1d738f', 'request_id': 'req-f5ce48fc-651b-4b57-ac60-988040706c55'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:12.779916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '378e3decbdca4e779c2b488b2a1d738f', 'request_id': 'req-f5ce48fc-651b-4b57-ac60-988040706c55'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:12.792315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] The token cache is being invalidate because project 378e3decbdca4e779c2b488b2a1d738f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:12.792604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:12.793284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:12.794917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1174/2349] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:12 2022] DELETE /identity/v3/projects/378e3decbdca4e779c2b488b2a1d738f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:12.798916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:12.813180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a785e6e-89a2-4508-89a3-8de1734d6e49 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:12.861488 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:12.876754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:12.877388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:12.877670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:12.877942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] PATH_INFO: `/v3/users/1cd109ac62ad4768b906d76c33ffd021` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:12.879094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd109ac62ad4768b906d76c33ffd021)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:12.881079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:13.058456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd109ac62ad4768b906d76c33ffd021'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.128897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd109ac62ad4768b906d76c33ffd021'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.129512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd109ac62ad4768b906d76c33ffd021'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.142032 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd109ac62ad4768b906d76c33ffd021'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.150000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72394e47-7b54-4e11-bdff-971d9d21c557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd109ac62ad4768b906d76c33ffd021'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.156175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1176/2350] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:12 2022] DELETE /identity/v3/users/1cd109ac62ad4768b906d76c33ffd021 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:13.580382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:13.581876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:13.595689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce48fc-651b-4b57-ac60-988040706c55 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:13.654967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:13.655580 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:13.655856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:13.656123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] PATH_INFO: `/v3/projects/b48af8a4bac64df2bab596137c7be7f6` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:13.657227 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48af8a4bac64df2bab596137c7be7f6)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:13.659516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:13.668285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:13.768143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48af8a4bac64df2bab596137c7be7f6', 'request_id': 'req-de6b128b-47ce-46e0-97a7-2fc452fb40f9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.836497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48af8a4bac64df2bab596137c7be7f6', 'request_id': 'req-de6b128b-47ce-46e0-97a7-2fc452fb40f9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.843056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48af8a4bac64df2bab596137c7be7f6', 'request_id': 'req-de6b128b-47ce-46e0-97a7-2fc452fb40f9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.854477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] The token cache is being invalidate because project b48af8a4bac64df2bab596137c7be7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:13.854771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.855411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-de6b128b-47ce-46e0-97a7-2fc452fb40f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:13.856901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1175/2351] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:13 2022] DELETE /identity/v3/projects/b48af8a4bac64df2bab596137c7be7f6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:13.862235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:13.862755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:13.862755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:13.883811 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:13.885012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:13.944679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1177/2352] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:13.953377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:13.953668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:13.953978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:13.973724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:13.975604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:13.983503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1176/2353] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:13.987661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:13.998028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2888f31-dbf3-43e5-82b7-a01fae3209b7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.002765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.004668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.004668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.004668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.004668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:39:14.005037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.006651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.018051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1178/2354] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:39:14.022320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.031664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7656c9-4b9f-49d2-b957-cb37fccfc28f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.036566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.037775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.038052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.038320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.038977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.040868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.158422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1177/2355] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:39:14 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:39:14.163219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.173025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcb352-6387-4ac8-bc38-0ce7a9ec7c2d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.220294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:14.229392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.229990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.230270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.230577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.231194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.233123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.322441 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1179/2356] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:39:14 2022] POST /identity/v3/users => generated 305 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:14.326610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.337232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3905e63-2265-4fbb-85a7-7c5f7dd41a49 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.341919 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.342543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.342823 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.343164 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.343799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.345422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.355749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1178/2357] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:14.359549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.370140 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c0ed-4263-4469-b84b-a38ed43640fe admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.374861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0317b-393b-45a8-b29c-06f237612530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.375449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0317b-393b-45a8-b29c-06f237612530 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.375725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0317b-393b-45a8-b29c-06f237612530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.375991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0317b-393b-45a8-b29c-06f237612530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.376621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0317b-393b-45a8-b29c-06f237612530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.378224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0317b-393b-45a8-b29c-06f237612530 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.389273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1180/2358] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:14.393209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.403707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f9064-9cae-451d-9b93-890e9493370e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.408820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.409409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.409686 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.409958 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] PATH_INFO: `/v3/projects/df1ffe4077c743d195c0a2a6001ed842/users/086027c22f3641cb946b6d02e345e858/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.425913 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1ffe4077c743d195c0a2a6001ed842, user_id=086027c22f3641cb946b6d02e345e858, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.428980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.482346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1179/2359] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:14 2022] PUT /identity/v3/projects/df1ffe4077c743d195c0a2a6001ed842/users/086027c22f3641cb946b6d02e345e858/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:14.492851 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.493142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.493420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.521062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] MFA Rules not processed for user `086027c22f3641cb946b6d02e345e858`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:14.523054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:14.585901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1181/2360] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:14 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:14.590384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.652100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408166d6-2257-4caa-9d93-3c14dc17b7ec admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.656738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.657337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.657617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.657887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.658817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.660780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:14.786380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1180/2361] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:39:14 2022] POST /identity/v3/projects => generated 409 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:14.790829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:14.801819 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75e7ff-f198-4e17-a4d0-35ace8139861 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:14.859064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:14.994788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:14.994788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:14.994788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:14.994788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:14.994788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.059696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1182/2362] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:39:14 2022] POST /identity/v3/users => generated 311 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:15.063707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:15.074642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d871fe-34db-4559-8b91-d9270581aee8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:15.079117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:15.079715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.079996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.080278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.080909 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:15.082580 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.094244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1181/2363] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:15.099994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:15.110956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f4a62-5c33-458b-afe0-e77b3e2a7984 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:15.115866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:15.116594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.116873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.117146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] PATH_INFO: `/v3/projects/e7fc7dced866457b983653983f6ea9ae/users/f9367e090dd646e9be56b0c610fd9701/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.132251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fc7dced866457b983653983f6ea9ae, user_id=f9367e090dd646e9be56b0c610fd9701, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:15.135200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.224687 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1183/2364] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:15 2022] PUT /identity/v3/projects/e7fc7dced866457b983653983f6ea9ae/users/f9367e090dd646e9be56b0c610fd9701/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:15.228581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:15.235063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:15.239057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc23bf9-df15-4226-98a3-810ba2afffa5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:15.297733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4845b-0189-4b4d-bff3-de4bc80a2f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:15.298358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4845b-0189-4b4d-bff3-de4bc80a2f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.298652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4845b-0189-4b4d-bff3-de4bc80a2f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.298924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4845b-0189-4b4d-bff3-de4bc80a2f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.299565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4845b-0189-4b4d-bff3-de4bc80a2f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:15.301154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4845b-0189-4b4d-bff3-de4bc80a2f38 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.311842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1182/2365] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:15.316099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:15.327573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961152-e4db-4d88-a71c-4058d1615392 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:15.332173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:15.333882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.333882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.333882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] PATH_INFO: `/v3/projects/e7fc7dced866457b983653983f6ea9ae/users/f9367e090dd646e9be56b0c610fd9701/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.335692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fc7dced866457b983653983f6ea9ae, user_id=f9367e090dd646e9be56b0c610fd9701, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:15.338477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.349392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1184/2366] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:15 2022] PUT /identity/v3/projects/e7fc7dced866457b983653983f6ea9ae/users/f9367e090dd646e9be56b0c610fd9701/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:15.359650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565e69-af6b-478b-9425-786f713524e2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.359942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565e69-af6b-478b-9425-786f713524e2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.360222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565e69-af6b-478b-9425-786f713524e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.389060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-97565e69-af6b-478b-9425-786f713524e2 None None] MFA Rules not processed for user `f9367e090dd646e9be56b0c610fd9701`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:15.390286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-97565e69-af6b-478b-9425-786f713524e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:15.458866 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1183/2367] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:15 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:39:15.469377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:15.471869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:15.487747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:15.531778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc81b99-99e9-42a2-a357-47444e41c743 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:15.535665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:15.536267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.536543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.536862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.546944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:15.548376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.558293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1185/2368] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:15 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:15.853460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565e69-af6b-478b-9425-786f713524e2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:15.862991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565e69-af6b-478b-9425-786f713524e2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:15.867235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbcb27-593f-40fd-9be9-23e9193d9ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:15.867834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcb27-593f-40fd-9be9-23e9193d9ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:15.868116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcb27-593f-40fd-9be9-23e9193d9ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:15.868389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbcb27-593f-40fd-9be9-23e9193d9ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:15.924752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcb27-593f-40fd-9be9-23e9193d9ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:15.926190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbcb27-593f-40fd-9be9-23e9193d9ae8 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:15.935323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1184/2369] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:15 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:25.554058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=20 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:25.931481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:29.283656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:29.297980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:29.311735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:29.311735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:29.326141 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:29.327645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:29.394709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290689a-8325-4cce-885a-d128eb0605b4 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:29.398703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1185/2370] 10.222.0.13 () {58 vars in 967 bytes} [Sat Sep 3 01:39:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:39:29.400174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:29.400908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:29.401223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:29.401354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] PATH_INFO: `/v3/users/086027c22f3641cb946b6d02e345e858` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:29.402714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=086027c22f3641cb946b6d02e345e858)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:29.404784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:29.539651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:29.541094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:29.610394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086027c22f3641cb946b6d02e345e858'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:29.611422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51836aa-dbb8-40c1-880e-3ab51e2c2a45 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:29.617599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb504a0-32b4-4830-8ad4-138568312ea1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:29.618217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb504a0-32b4-4830-8ad4-138568312ea1 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:29.623282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb504a0-32b4-4830-8ad4-138568312ea1 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:29.623282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb504a0-32b4-4830-8ad4-138568312ea1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:29.628586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb504a0-32b4-4830-8ad4-138568312ea1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:29.630276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb504a0-32b4-4830-8ad4-138568312ea1 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:29.638954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1186/2371] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:39:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:39:29.687244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086027c22f3641cb946b6d02e345e858'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:29.687868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086027c22f3641cb946b6d02e345e858'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:29.702751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086027c22f3641cb946b6d02e345e858'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:29.709790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086027c22f3641cb946b6d02e345e858'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:29.716336 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1186/2372] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:29 2022] DELETE /identity/v3/users/086027c22f3641cb946b6d02e345e858 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:29.721058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d161-185a-4273-8e82-bac358680586 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:29.721339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d161-185a-4273-8e82-bac358680586 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:29.721608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d161-185a-4273-8e82-bac358680586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:29.742762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-18d1d161-185a-4273-8e82-bac358680586 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:29.744037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-18d1d161-185a-4273-8e82-bac358680586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:29.806228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1187/2373] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:29.815858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:29.817271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:29.880438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3f4e1-3dc8-4681-acc4-4848b59d4c47 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:29.884143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:29.884752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:29.885040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:29.885315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:29.894152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:29.895636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:29.905149 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1187/2374] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:30.492126 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d161-185a-4273-8e82-bac358680586 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:30.494574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-18d1d161-185a-4273-8e82-bac358680586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:30.524271 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d161-185a-4273-8e82-bac358680586 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:30.536106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:30.537339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:30.537718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:30.538078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] PATH_INFO: `/v3/projects/df1ffe4077c743d195c0a2a6001ed842` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:30.539482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1ffe4077c743d195c0a2a6001ed842)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:30.541497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:30.718649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:30.737746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1ffe4077c743d195c0a2a6001ed842', 'request_id': 'req-275af480-bbc0-4bb7-9b98-c8050cec79e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:30.824817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1ffe4077c743d195c0a2a6001ed842', 'request_id': 'req-275af480-bbc0-4bb7-9b98-c8050cec79e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:30.835870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1ffe4077c743d195c0a2a6001ed842', 'request_id': 'req-275af480-bbc0-4bb7-9b98-c8050cec79e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:30.851325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] The token cache is being invalidate because project df1ffe4077c743d195c0a2a6001ed842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:30.851625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:30.852498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:30.854389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1188/2375] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:30 2022] DELETE /identity/v3/projects/df1ffe4077c743d195c0a2a6001ed842 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:30.859146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:30.871538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5ecf0-ebfd-405b-a42a-add682ae8213 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:30.903115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:30.959933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:30.961031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:30.961401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:30.961740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] PATH_INFO: `/v3/users/f9367e090dd646e9be56b0c610fd9701` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:30.963180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9367e090dd646e9be56b0c610fd9701)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:30.965201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:31.151566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9367e090dd646e9be56b0c610fd9701'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:31.207171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9367e090dd646e9be56b0c610fd9701'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:31.208080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9367e090dd646e9be56b0c610fd9701'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:31.222712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9367e090dd646e9be56b0c610fd9701'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:31.231698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f835d45f-7670-4781-9d95-73f76c04c292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9367e090dd646e9be56b0c610fd9701'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:31.240716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1188/2376] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:30 2022] DELETE /identity/v3/users/f9367e090dd646e9be56b0c610fd9701 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:31.917949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:31.919466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:31.933431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275af480-bbc0-4bb7-9b98-c8050cec79e4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.014192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.020420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.020420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.020420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] PATH_INFO: `/v3/projects/e7fc7dced866457b983653983f6ea9ae` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.020420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7fc7dced866457b983653983f6ea9ae)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.022063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:32.171851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7fc7dced866457b983653983f6ea9ae', 'request_id': 'req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:32.247849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7fc7dced866457b983653983f6ea9ae', 'request_id': 'req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:32.255496 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7fc7dced866457b983653983f6ea9ae', 'request_id': 'req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:32.276538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] The token cache is being invalidate because project e7fc7dced866457b983653983f6ea9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:32.276851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:32.277493 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-55ccd49f-5ab1-4e61-9b6d-79eb2323fa67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:32.279118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1189/2377] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:31 2022] DELETE /identity/v3/projects/e7fc7dced866457b983653983f6ea9ae => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:32.285945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.286529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.286615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.309190 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:32.310443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:32.396986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1189/2378] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:32.405506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866830b-680d-414e-9dea-5187027c6777 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.405796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866830b-680d-414e-9dea-5187027c6777 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.406090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866830b-680d-414e-9dea-5187027c6777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.428416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b866830b-680d-414e-9dea-5187027c6777 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:32.430453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b866830b-680d-414e-9dea-5187027c6777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:32.438917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1190/2379] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:32.451011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:32.460984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e667b-e3db-4794-81fa-757e4c763da9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.465583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.466186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.466520 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.466795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.467458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:39:32.467760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.469436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:32.481698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1190/2380] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:39:32.489861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866830b-680d-414e-9dea-5187027c6777 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:32.509666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866830b-680d-414e-9dea-5187027c6777 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.514849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.515474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.515747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.516021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.516856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.518812 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:32.604621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1191/2381] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:39:32 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:32.608834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:32.626243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad12b6-2701-4ac0-a417-1b448e463a57 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.692890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:32.748348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.749308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.750529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.751989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.752983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.756142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:32.842766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1191/2382] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:39:32 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:32.843456 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:32.857918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b956a-f42e-4069-ba9d-94c3602a620a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.863514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.874058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.874058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.874058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.874058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.874058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:32.878748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1192/2383] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:32.884455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:32.896406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503edfa4-b346-4559-bc11-fdee588403ff admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.903538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8697-86d5-4ee8-90e0-562cf61879a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.904522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8697-86d5-4ee8-90e0-562cf61879a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.904931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8697-86d5-4ee8-90e0-562cf61879a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.905323 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8697-86d5-4ee8-90e0-562cf61879a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.906202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8697-86d5-4ee8-90e0-562cf61879a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.907935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb8697-86d5-4ee8-90e0-562cf61879a0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:32.927607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1192/2384] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:32.928875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:32.944348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5a1af-fe07-459e-bc83-4172657e24c7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:32.951475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:32.953923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:32.954615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:32.955108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] PATH_INFO: `/v3/projects/3ea84ca91c7345b294a0fbdb77ee1799/users/68d6c1fd6f724877b708b3c812ccf944/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:32.961318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:32.980832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea84ca91c7345b294a0fbdb77ee1799, user_id=68d6c1fd6f724877b708b3c812ccf944, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:32.984548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:33.084204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1193/2385] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:32 2022] PUT /identity/v3/projects/3ea84ca91c7345b294a0fbdb77ee1799/users/68d6c1fd6f724877b708b3c812ccf944/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:39:33.097544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:33.097891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:33.098221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:33.159837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] MFA Rules not processed for user `68d6c1fd6f724877b708b3c812ccf944`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:33.161145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:33.244721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1193/2386] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:33.259666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:33.261732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:33.349959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1837d-2746-4780-a9f6-6ef06c8dbb58 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:33.357425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37c059-1f10-457a-a2d5-1abcdae10102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:33.357920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c059-1f10-457a-a2d5-1abcdae10102 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:33.358208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c059-1f10-457a-a2d5-1abcdae10102 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:33.358536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37c059-1f10-457a-a2d5-1abcdae10102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:33.371539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37c059-1f10-457a-a2d5-1abcdae10102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:33.373267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37c059-1f10-457a-a2d5-1abcdae10102 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:33.385643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1194/2387] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:33.412113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:33.507132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53532017-d695-48ee-93c9-7b4d8049f5d5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:33.512526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:33.513285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:33.513572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:33.513849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] PATH_INFO: `/v3/users/68d6c1fd6f724877b708b3c812ccf944` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:33.515042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d6c1fd6f724877b708b3c812ccf944)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:33.517491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:33.721109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:33.733116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d6c1fd6f724877b708b3c812ccf944'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:33.807713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d6c1fd6f724877b708b3c812ccf944'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:33.808581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d6c1fd6f724877b708b3c812ccf944'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:33.827568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d6c1fd6f724877b708b3c812ccf944'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:33.836319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d6c1fd6f724877b708b3c812ccf944'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:33.847764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1194/2388] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:33 2022] DELETE /identity/v3/users/68d6c1fd6f724877b708b3c812ccf944 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:33.852714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:33.853072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:33.853465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:33.889644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:33.891273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:34.025555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1195/2389] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:34.051136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:34.059885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:34.167326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10680771-ac25-4a5e-9c2f-bad8f3843262 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:34.173031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e08a05-125e-4ed4-864d-ba2498ea8911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:34.174155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08a05-125e-4ed4-864d-ba2498ea8911 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:34.175094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08a05-125e-4ed4-864d-ba2498ea8911 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:34.175782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e08a05-125e-4ed4-864d-ba2498ea8911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:34.197237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e08a05-125e-4ed4-864d-ba2498ea8911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:34.199438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e08a05-125e-4ed4-864d-ba2498ea8911 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:34.211109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1195/2390] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:34.898766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:34.900509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:34.922727 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5c07e-8177-45fc-bd30-91061b86b7f1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:34.949448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:34.954551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:34.954551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:34.954551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] PATH_INFO: `/v3/projects/3ea84ca91c7345b294a0fbdb77ee1799` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:34.954551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea84ca91c7345b294a0fbdb77ee1799)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:34.954551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.040200 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b126-d734-4b1b-8de1-f988d1358321 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.040720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b126-d734-4b1b-8de1-f988d1358321 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.040720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b126-d734-4b1b-8de1-f988d1358321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.060915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b122b126-d734-4b1b-8de1-f988d1358321 None None] MFA Rules not processed for user `2b9ab1983bb84318b509878b930e9402`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:35.062130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b122b126-d734-4b1b-8de1-f988d1358321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:35.095112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea84ca91c7345b294a0fbdb77ee1799', 'request_id': 'req-def244cb-08a0-478a-b618-51ff986a836d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:35.138557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1196/2391] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:35.161495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea84ca91c7345b294a0fbdb77ee1799', 'request_id': 'req-def244cb-08a0-478a-b618-51ff986a836d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:35.170214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea84ca91c7345b294a0fbdb77ee1799', 'request_id': 'req-def244cb-08a0-478a-b618-51ff986a836d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:35.184839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] The token cache is being invalidate because project 3ea84ca91c7345b294a0fbdb77ee1799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:35.184951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:35.185607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-def244cb-08a0-478a-b618-51ff986a836d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:35.187754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1196/2392] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:34 2022] DELETE /identity/v3/projects/3ea84ca91c7345b294a0fbdb77ee1799 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:35.192353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.192629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.192904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.224692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:35.226508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:35.298457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1197/2393] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:35.307950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.308242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.308516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.334217 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:35.336381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:35.345187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1197/2394] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:35.349909 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:35.359034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:35.369433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54111-cfaa-475c-ab0e-675e3f0a1058 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:35.380221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:35.380221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.380221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.380221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.380831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:39:35.380831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:35.380831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.393646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1198/2395] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:39:35.398713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:35.405570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:35.410129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19c0a9-703e-4cbf-bf36-196cf6b4c351 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:35.416810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:35.417506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.417848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.418191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.418952 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:35.420891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.531348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1198/2396] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:39:35 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:39:35.531484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:35.544584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c01f5-abd3-402e-ab13-f84e2a56d874 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:35.618436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:35.619102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.619391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.619666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.620307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:35.622301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.718386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1199/2397] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:39:35 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:39:35.722672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:35.734256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a385b-97c0-4378-b36a-caefd18c23e6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:35.741247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:35.743107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.743401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.743678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.744343 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:35.745966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.761139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1199/2398] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:39:35.762254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:35.775893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55b374-7264-43c7-ba67-22cd841cee67 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:35.785091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671b63c-ac45-4d77-8b1b-87263806eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:35.786094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671b63c-ac45-4d77-8b1b-87263806eba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.786593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671b63c-ac45-4d77-8b1b-87263806eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.787071 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671b63c-ac45-4d77-8b1b-87263806eba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.789538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671b63c-ac45-4d77-8b1b-87263806eba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:35.792427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671b63c-ac45-4d77-8b1b-87263806eba8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.812694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1200/2399] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:39:35.813921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:35.831719 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4851df7-f755-4082-8783-c2c8a304c724 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:35.838495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:35.839194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.839479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.839760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] PATH_INFO: `/v3/projects/8fcd5e5873994e3a9d3e746c8ce9f8dc/users/6d25341b48e145ae8439be325e8c4e5a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:35.864742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcd5e5873994e3a9d3e746c8ce9f8dc, user_id=6d25341b48e145ae8439be325e8c4e5a, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:35.870435 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:35.971094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1200/2400] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:39:35 2022] PUT /identity/v3/projects/8fcd5e5873994e3a9d3e746c8ce9f8dc/users/6d25341b48e145ae8439be325e8c4e5a/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:39:35.981227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:35.981521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:35.981793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:36.046062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] MFA Rules not processed for user `6d25341b48e145ae8439be325e8c4e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:36.046062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:36.144410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1201/2401] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:36.156096 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:36.157845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:36.218515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847eb1d1-8cae-41eb-a1a6-61bb2f17cac5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:36.224127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:36.224127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:36.224127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:36.224127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:36.232798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:36.234246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:36.243986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1201/2402] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:40.161998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:40.164028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:40.238322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:40.243836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d034a85-652d-44b2-a668-b941af079c34 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:40.248761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:40.249364 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:40.249643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:40.249939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:40.251766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:40.263477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:40.265058 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:40.275473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1202/2403] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:40.310973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:40.310973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:40.377740 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed300cab-de2d-4f14-8c43-f65ec9e7df43 service glance] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:40.383216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:40.383649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:40.383928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:40.384201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] PATH_INFO: `/v3/users/b8349bd45ad84f14858f73bf42b48380` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:40.385368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8349bd45ad84f14858f73bf42b48380)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:40.387358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:40.530557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8349bd45ad84f14858f73bf42b48380'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:40.531253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8349bd45ad84f14858f73bf42b48380'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:40.536877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:40.545378 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8349bd45ad84f14858f73bf42b48380'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:40.555341 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8349bd45ad84f14858f73bf42b48380'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:40.606515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8349bd45ad84f14858f73bf42b48380'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:40.613418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1202/2404] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:40 2022] DELETE /identity/v3/users/b8349bd45ad84f14858f73bf42b48380 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:40.893175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:40.911945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be421a48-16bc-4fe0-b9d5-d6f116a76cb2 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:40.971035 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:40.971680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:40.971958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:40.972228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] PATH_INFO: `/v3/projects/364812df6abf492698453972e9a1d120` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:40.973343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=364812df6abf492698453972e9a1d120)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:40.975313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:41.138139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364812df6abf492698453972e9a1d120', 'request_id': 'req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.201736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364812df6abf492698453972e9a1d120', 'request_id': 'req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.207867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364812df6abf492698453972e9a1d120', 'request_id': 'req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.219062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] The token cache is being invalidate because project 364812df6abf492698453972e9a1d120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:41.219361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.219986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.221469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1203/2405] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:40 2022] DELETE /identity/v3/projects/364812df6abf492698453972e9a1d120 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:41.226130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:41.227761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:41.240894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb62286-086b-4042-9ec5-8d02aa0a8311 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:41.307477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:41.308108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:41.308382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:41.308649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] PATH_INFO: `/v3/users/2882eb54386c48be8186b53ea3ce2857` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:41.309765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2882eb54386c48be8186b53ea3ce2857)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:41.311717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:41.495335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2882eb54386c48be8186b53ea3ce2857'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.496197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2882eb54386c48be8186b53ea3ce2857'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.514947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2882eb54386c48be8186b53ea3ce2857'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.521977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2882eb54386c48be8186b53ea3ce2857'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.675622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2882eb54386c48be8186b53ea3ce2857'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:41.694978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1203/2406] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:41 2022] DELETE /identity/v3/users/2882eb54386c48be8186b53ea3ce2857 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:41.957791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:41.968335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:41.977123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0452f-7e79-4ded-9ae7-0384a6a180ee admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:42.056074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:42.056754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:42.057037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:42.057340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] PATH_INFO: `/v3/projects/ef464bb9a5c44389851e147a8fd2866d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:42.058675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef464bb9a5c44389851e147a8fd2866d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:42.060618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:42.087392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:42.171256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef464bb9a5c44389851e147a8fd2866d', 'request_id': 'req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:42.230076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef464bb9a5c44389851e147a8fd2866d', 'request_id': 'req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:42.236886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef464bb9a5c44389851e147a8fd2866d', 'request_id': 'req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:42.249834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] The token cache is being invalidate because project ef464bb9a5c44389851e147a8fd2866d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:42.250425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:42.251245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8d3d3c01-a8b2-4499-8b0f-fae7289a6760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:42.256483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1204/2407] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:41 2022] DELETE /identity/v3/projects/ef464bb9a5c44389851e147a8fd2866d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:51.678867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=13,keystone:SELECT=63,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:52.556197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:39:58.569258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:58.571497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:58.595371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d0c2b-6443-406f-8705-2e3b2c611999 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:58.658489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:58.659499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:58.659832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:58.660175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] PATH_INFO: `/v3/users/6d25341b48e145ae8439be325e8c4e5a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:58.661515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d25341b48e145ae8439be325e8c4e5a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:58.663484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:58.852093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d25341b48e145ae8439be325e8c4e5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:58.853275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d25341b48e145ae8439be325e8c4e5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:58.868887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d25341b48e145ae8439be325e8c4e5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:58.875940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d25341b48e145ae8439be325e8c4e5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:58.989798 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d25341b48e145ae8439be325e8c4e5a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:58.996830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1204/2408] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:39:58 2022] DELETE /identity/v3/users/6d25341b48e145ae8439be325e8c4e5a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:59.003182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:59.003706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:59.003706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:59.025053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:59.026362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:59.091338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1205/2409] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:39:59.098100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:59.099680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:59.172636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e757-7cb4-48ef-8f26-896b63e2fe91 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:59.176357 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1850f-36b5-430a-8996-4910713c9ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:59.177047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1850f-36b5-430a-8996-4910713c9ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:59.177393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1850f-36b5-430a-8996-4910713c9ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:59.177700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1850f-36b5-430a-8996-4910713c9ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:59.186722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1850f-36b5-430a-8996-4910713c9ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:59.188165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1850f-36b5-430a-8996-4910713c9ef8 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:59.198515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1205/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:39:59.625524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:39:59.627109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:39:59.640937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d3f8f-1c2f-4472-a565-48900735bf82 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:39:59.645712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:39:59.650475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:59.650475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:59.650475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] PATH_INFO: `/v3/projects/8fcd5e5873994e3a9d3e746c8ce9f8dc` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:59.650475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcd5e5873994e3a9d3e746c8ce9f8dc)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:39:59.650475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:39:59.832559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcd5e5873994e3a9d3e746c8ce9f8dc', 'request_id': 'req-9ab3f947-063a-4e62-930f-ccdf8f1f795e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:59.943900 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcd5e5873994e3a9d3e746c8ce9f8dc', 'request_id': 'req-9ab3f947-063a-4e62-930f-ccdf8f1f795e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:59.950324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcd5e5873994e3a9d3e746c8ce9f8dc', 'request_id': 'req-9ab3f947-063a-4e62-930f-ccdf8f1f795e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:59.961783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] The token cache is being invalidate because project 8fcd5e5873994e3a9d3e746c8ce9f8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:39:59.962152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:59.962872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9ab3f947-063a-4e62-930f-ccdf8f1f795e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:39:59.964349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1206/2411] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:39:59 2022] DELETE /identity/v3/projects/8fcd5e5873994e3a9d3e746c8ce9f8dc => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:39:59.969228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:39:59.972899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:39:59.972899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:39:59.989843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:39:59.991114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:00.008519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:00.051483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1206/2412] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:00.059295 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.059895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.059895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.080109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:00.081238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:00.090077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1207/2413] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:00.094267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.104571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56866bbf-eb9e-46ad-8621-fcfb7078713f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.109079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.109672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.109946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.110210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.110891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:40:00.111216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.112788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.124225 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1207/2414] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:40:00.128578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.137812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e18d1c-4873-4f70-aa08-6a5702c7e8dd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.142513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.143135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.143415 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.143680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.144314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.146220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.281051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1208/2415] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:00 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:00.282470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.293764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc6452-6def-4835-a4cd-b2678f1b79a6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.357482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.358089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.358402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.358677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.359303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.361397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.477165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1208/2416] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:00 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:40:00.480268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.490562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130975d-7f35-468e-a4af-116a59cd7f6a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.495383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.496002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.500644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.500644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.500644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.500644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.509338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1209/2417] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:00.513320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.523354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b9f57-9b2b-4050-9740-a01578e9fe29 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.528773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64718e0b-e25f-4207-aae1-4de556d82c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.529363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64718e0b-e25f-4207-aae1-4de556d82c30 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.529639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64718e0b-e25f-4207-aae1-4de556d82c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.529905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64718e0b-e25f-4207-aae1-4de556d82c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.530682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64718e0b-e25f-4207-aae1-4de556d82c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.532265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64718e0b-e25f-4207-aae1-4de556d82c30 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.546931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1209/2418] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:00.555321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.565567 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942e00f-9c5d-4efe-b38a-50861b967bfc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.570459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.571946 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.572371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.572650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] PATH_INFO: `/v3/projects/ec648ae5ff724a9d9e011e0abc58f197/users/77ccff548bac483fb09c77c2cc7bc397/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.589810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec648ae5ff724a9d9e011e0abc58f197, user_id=77ccff548bac483fb09c77c2cc7bc397, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.592667 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.666313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1210/2419] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:00 2022] PUT /identity/v3/projects/ec648ae5ff724a9d9e011e0abc58f197/users/77ccff548bac483fb09c77c2cc7bc397/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:00.732498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.732791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.733056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.761452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] MFA Rules not processed for user `77ccff548bac483fb09c77c2cc7bc397`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:00.762707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:00.838124 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1210/2420] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:00.849613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:00.851753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:00.874313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:00.911045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5168ee-e121-47b2-bbec-3a6ba0cc737e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:00.915119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:00.915755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:00.916038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:00.916312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:00.924854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:00.926509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:00.944671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1211/2421] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:03.740856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:03.742670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:03.767997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:03.821594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481d69c-54c2-4230-aff6-102cc8724469 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:03.826531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:03.827172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:03.827455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:03.827722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] PATH_INFO: `/v3/users/69028bf5296541b6bfe9595d684c6567` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:03.828969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69028bf5296541b6bfe9595d684c6567)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:03.830962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:04.015764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69028bf5296541b6bfe9595d684c6567'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.016581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69028bf5296541b6bfe9595d684c6567'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.030149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69028bf5296541b6bfe9595d684c6567'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.037334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69028bf5296541b6bfe9595d684c6567'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.099235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:04.110683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428673a9-e3ff-4f9b-a7a7-fec73be2cf9a service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:04.115484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:04.116147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:04.116430 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:04.116699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] PATH_INFO: `/v3/users/77ccff548bac483fb09c77c2cc7bc397` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:04.117853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ccff548bac483fb09c77c2cc7bc397)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:04.119911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:04.132432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8fe5014c-c722-4253-8376-5708ad98ed76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69028bf5296541b6bfe9595d684c6567'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.141248 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1211/2422] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:03 2022] DELETE /identity/v3/users/69028bf5296541b6bfe9595d684c6567 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:04.308300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ccff548bac483fb09c77c2cc7bc397'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.379942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ccff548bac483fb09c77c2cc7bc397'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.380688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ccff548bac483fb09c77c2cc7bc397'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.394887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ccff548bac483fb09c77c2cc7bc397'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.402405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ccff548bac483fb09c77c2cc7bc397'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.408387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1212/2423] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:04 2022] DELETE /identity/v3/users/77ccff548bac483fb09c77c2cc7bc397 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:04.413197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:04.414918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:04.414918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:04.433934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:04.435215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:04.466227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:04.467736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:04.509112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1212/2424] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:04.516163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:04.518033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:04.538543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd5c76-6347-455f-b2d0-f0ea79829765 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:04.544002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:04.544643 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:04.544928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:04.545207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] PATH_INFO: `/v3/projects/efb12fa605334996a0e25888c90a108a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:04.546377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb12fa605334996a0e25888c90a108a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:04.548307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:04.561984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:04.577616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61bfae-966c-4550-8310-9c022f140aae None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:04.582186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:04.582830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:04.583115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:04.583382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:04.593353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:04.594959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:04.605886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1213/2425] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:40:04.702837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:04.712986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb12fa605334996a0e25888c90a108a', 'request_id': 'req-7a2952e7-eeb8-4201-aef6-64fde9995958'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.790942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb12fa605334996a0e25888c90a108a', 'request_id': 'req-7a2952e7-eeb8-4201-aef6-64fde9995958'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.805017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb12fa605334996a0e25888c90a108a', 'request_id': 'req-7a2952e7-eeb8-4201-aef6-64fde9995958'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.817427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] The token cache is being invalidate because project efb12fa605334996a0e25888c90a108a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:04.817785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.818543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a2952e7-eeb8-4201-aef6-64fde9995958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:04.822207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1213/2426] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:04 2022] DELETE /identity/v3/projects/efb12fa605334996a0e25888c90a108a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:05.151817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:05.153964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:05.179682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91044cda-9fab-45f5-8371-9230f589d3b4 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:05.239057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:05.239735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:05.240011 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:05.240283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] PATH_INFO: `/v3/projects/ec648ae5ff724a9d9e011e0abc58f197` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:05.241390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec648ae5ff724a9d9e011e0abc58f197)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:05.243598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:05.333233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec648ae5ff724a9d9e011e0abc58f197', 'request_id': 'req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:05.429574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec648ae5ff724a9d9e011e0abc58f197', 'request_id': 'req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:05.436170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec648ae5ff724a9d9e011e0abc58f197', 'request_id': 'req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:05.448085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] The token cache is being invalidate because project ec648ae5ff724a9d9e011e0abc58f197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:05.448440 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:05.449151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:05.450673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1214/2427] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:05 2022] DELETE /identity/v3/projects/ec648ae5ff724a9d9e011e0abc58f197 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:09.746810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:09.747309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:09.748010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:09.768344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:09.769564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:09.826250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1214/2428] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:40:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:09.838131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:09.839601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:09.907579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1bdd-01e1-49cb-af95-e0aac7d3e21b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:09.913025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:09.913644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:09.913928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:09.914221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:09.922473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:09.924051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:09.928622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:09.932967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1215/2429] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:40:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:40:19.818704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=36,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:25.579752 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:25.581886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:25.646593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b63a6-d090-4343-bfb7-fbb7ebb48e48 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:25.651180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:25.651890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:25.652234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:25.652587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:25.654538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:25.719536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:25.721023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:25.732432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1215/2430] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:29.841323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:29.843689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:29.934182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d4092-ef49-433a-bb55-759ca69f84ed service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:29.939771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:29.940553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:29.940951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:29.941329 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:29.943365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:29.957059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:29.958881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605cb7cd-4d7f-49c6-9296-7d199de05f35 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:29.978552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1216/2431] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:30.036732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:30.039367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:30.054574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845861a-cd00-45ba-9dd9-a855cc7d1ea9 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:30.060349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:30.060987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:30.061425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:30.061874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] PATH_INFO: `/v3/users/ecf076dde874415aa734644f27a10b4b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:30.063301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf076dde874415aa734644f27a10b4b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:30.067113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:30.309921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf076dde874415aa734644f27a10b4b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:30.367274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf076dde874415aa734644f27a10b4b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:30.368145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf076dde874415aa734644f27a10b4b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:30.388066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf076dde874415aa734644f27a10b4b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:30.390670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:30.390918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:30.391194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:30.396698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf076dde874415aa734644f27a10b4b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:30.403383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1216/2432] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:30 2022] DELETE /identity/v3/users/ecf076dde874415aa734644f27a10b4b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:30.421147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:30.423232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:30.500186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1217/2433] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:40:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:40:30.523754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:30.525594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:30.573887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:30.654493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555a250-ccc2-4c01-a730-0b89d3e72e9d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:30.672062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4be973-e1cc-4e6a-b3d7-303c1b4af11e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:30.674018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4be973-e1cc-4e6a-b3d7-303c1b4af11e service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:30.674737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4be973-e1cc-4e6a-b3d7-303c1b4af11e service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:30.680375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4be973-e1cc-4e6a-b3d7-303c1b4af11e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:30.698577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4be973-e1cc-4e6a-b3d7-303c1b4af11e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:30.700376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4be973-e1cc-4e6a-b3d7-303c1b4af11e service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:30.710369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1217/2434] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:40:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:40:30.997627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:30.999116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:31.106409 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe6bf8-cd92-45e7-8dfc-0a3158af581c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:31.112725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:31.113651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.114099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.115384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] PATH_INFO: `/v3/projects/53cde5b369084772897d46c38b083d8f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.120070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=53cde5b369084772897d46c38b083d8f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:31.122022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:31.234994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:31.244885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53cde5b369084772897d46c38b083d8f', 'request_id': 'req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:31.330898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ffd5e-cd41-4b83-aa31-244e32bc8298 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.331378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ffd5e-cd41-4b83-aa31-244e32bc8298 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.331618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ffd5e-cd41-4b83-aa31-244e32bc8298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.337158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53cde5b369084772897d46c38b083d8f', 'request_id': 'req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:31.345818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53cde5b369084772897d46c38b083d8f', 'request_id': 'req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:31.359266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-eb1ffd5e-cd41-4b83-aa31-244e32bc8298 None None] MFA Rules not processed for user `df0ffa9f450642cc94ed828cb34034b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:31.361255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eb1ffd5e-cd41-4b83-aa31-244e32bc8298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:31.367025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] The token cache is being invalidate because project 53cde5b369084772897d46c38b083d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:31.367713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:31.369605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e8e6d115-a7d0-43a3-a2c5-ce0ad4ca29bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:31.372464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1218/2435] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:30 2022] DELETE /identity/v3/projects/53cde5b369084772897d46c38b083d8f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:31.377632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.378052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.378533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.407167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:31.409762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:31.441950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1218/2436] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:31.505542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1219/2437] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:31.506702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.507053 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.507325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.545266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:31.546541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:31.562668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1219/2438] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:31.566077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:31.581288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccc860-91a9-4ae5-bbfe-2cf4b599f15f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:31.586768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:31.587604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.588039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.588450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.589246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:40:31.589941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:31.591767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:31.607425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1220/2439] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:40:31.611454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:31.623029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7e30f-1e8c-449e-9571-4bd38b840a14 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:31.640763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:31.641782 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.642791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.643237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.644046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:31.646134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:31.762179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1220/2440] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:31 2022] POST /identity/v3/projects => generated 405 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:40:31.763807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:31.780865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b708e-ee6b-4a09-a175-653f4d215621 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:31.898397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:31.899381 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:31.899495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:31.899794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:31.900498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:31.902665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:31.986373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1221/2441] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:31 2022] POST /identity/v3/users => generated 303 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:31.988934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.004720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3518d88-948e-4892-bf8e-c0f8bad1c75c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.009882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.010546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.010905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.011184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.011979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.013854 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.024759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1221/2442] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:32.029015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.043915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb9c36-469a-448a-8e0e-f461eae48060 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.049327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e2963-9806-4215-b492-de24820d198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.049944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e2963-9806-4215-b492-de24820d198b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.050219 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e2963-9806-4215-b492-de24820d198b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.050518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e2963-9806-4215-b492-de24820d198b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.051234 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e2963-9806-4215-b492-de24820d198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.052841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e2963-9806-4215-b492-de24820d198b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.068352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1222/2443] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:32.072816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.083645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320efcb1-84bd-4ca0-b32b-946f2aedf8e6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.092431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.093203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.093475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.093749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] PATH_INFO: `/v3/projects/461990bc9adb4dcf9c815e8230469063/users/11fc152fca6a4f0eb812ef64e85bbf12/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.110255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461990bc9adb4dcf9c815e8230469063, user_id=11fc152fca6a4f0eb812ef64e85bbf12, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.112977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.196963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1222/2444] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:32 2022] PUT /identity/v3/projects/461990bc9adb4dcf9c815e8230469063/users/11fc152fca6a4f0eb812ef64e85bbf12/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:32.210502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.214706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.214706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.253360 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] MFA Rules not processed for user `11fc152fca6a4f0eb812ef64e85bbf12`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:32.254723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:32.282925 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:32.343110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1223/2445] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:32 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:32.344276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.422631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:32.438514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac81e9a-fda8-4823-9691-fb1df3f36cb0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.443330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.444134 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.445081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.449935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.451029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.453795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.550358 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1223/2446] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:32 2022] POST /identity/v3/projects => generated 405 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:32.553800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.569275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12ee8c-a861-4cf1-a41b-5751fb25d2ad None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.639002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.639755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.640040 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.640315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.640957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.642945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.751697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1224/2447] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:32 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:32.755942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.768471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c332563-022e-49f8-8d49-0cf41f3af42f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.777521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.778241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.778699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.779068 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.780025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.781795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.794250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1224/2448] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:40:32.798399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.815912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed66a4-56a1-4883-b115-0f56d858c64a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:32.825810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:32.825810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:32.825810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:32.825810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] PATH_INFO: `/v3/projects/fa2e7edacbc7428f978047bd77ae1b67/users/91c3f809f7d5464d9021c0b00e513d5f/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:32.840674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2e7edacbc7428f978047bd77ae1b67, user_id=91c3f809f7d5464d9021c0b00e513d5f, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:32.845182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:32.942021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1225/2449] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:32 2022] PUT /identity/v3/projects/fa2e7edacbc7428f978047bd77ae1b67/users/91c3f809f7d5464d9021c0b00e513d5f/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:32.946626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:32.970019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515e3c1-88f1-4c9d-ae2f-31dbc6526535 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:33.117613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c2e90-b06c-447b-8af1-d0884a0a1d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:33.119234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c2e90-b06c-447b-8af1-d0884a0a1d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:33.119234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c2e90-b06c-447b-8af1-d0884a0a1d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:33.119465 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c2e90-b06c-447b-8af1-d0884a0a1d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:33.119938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c2e90-b06c-447b-8af1-d0884a0a1d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:33.121610 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c2e90-b06c-447b-8af1-d0884a0a1d0d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:33.146723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1225/2450] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:33.151889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:33.177546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e2661-2d55-4760-b7d7-5590b1f79e36 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:33.179697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:33.192851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:33.193261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:33.193643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] PATH_INFO: `/v3/projects/fa2e7edacbc7428f978047bd77ae1b67/users/91c3f809f7d5464d9021c0b00e513d5f/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:33.196645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2e7edacbc7428f978047bd77ae1b67, user_id=91c3f809f7d5464d9021c0b00e513d5f, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:33.199494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:33.220514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1226/2451] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:33 2022] PUT /identity/v3/projects/fa2e7edacbc7428f978047bd77ae1b67/users/91c3f809f7d5464d9021c0b00e513d5f/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:33.231565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:33.231855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:33.232130 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:33.283400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] MFA Rules not processed for user `91c3f809f7d5464d9021c0b00e513d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:33.285276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:33.387245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1226/2452] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:33.398301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:33.400299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:33.497336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45501f3-37b5-4ef1-a3c2-a5389bbefd6b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:33.501049 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b921c8-7971-4dc1-9ce3-79a21e56123c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:33.501672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b921c8-7971-4dc1-9ce3-79a21e56123c service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:33.501954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b921c8-7971-4dc1-9ce3-79a21e56123c service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:33.502222 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b921c8-7971-4dc1-9ce3-79a21e56123c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:33.512824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b921c8-7971-4dc1-9ce3-79a21e56123c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:33.514530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b921c8-7971-4dc1-9ce3-79a21e56123c service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:33.524901 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1227/2453] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:33.720594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:33.814675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:33.837509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2b4e2-3be0-4ca1-ad72-df064df486ae None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:33.844457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:33.845470 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:33.845866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:33.846211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] PATH_INFO: `/v3/users/11fc152fca6a4f0eb812ef64e85bbf12` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:33.848125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11fc152fca6a4f0eb812ef64e85bbf12)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:33.850809 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:34.050847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11fc152fca6a4f0eb812ef64e85bbf12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:34.162642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11fc152fca6a4f0eb812ef64e85bbf12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:34.165908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11fc152fca6a4f0eb812ef64e85bbf12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:34.181135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11fc152fca6a4f0eb812ef64e85bbf12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:34.206987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11fc152fca6a4f0eb812ef64e85bbf12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:34.234681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1227/2454] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:33 2022] DELETE /identity/v3/users/11fc152fca6a4f0eb812ef64e85bbf12 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:34.235015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:34.235015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:34.235015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:34.273829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:34.275074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:34.308411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:34.400162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1228/2455] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:34.410661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:34.412143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:34.540229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965ea58-e61b-44aa-a489-a495112dc790 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:34.552253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:34.552941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:34.553223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:34.553497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:34.566155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:34.567753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:34.580037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1228/2456] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:35.299516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:35.300855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:35.314876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7d60-0d0c-4a21-b16f-dc0c95ed42c7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:35.324387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:35.325384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:35.325771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:35.326150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] PATH_INFO: `/v3/projects/461990bc9adb4dcf9c815e8230469063` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:35.327928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=461990bc9adb4dcf9c815e8230469063)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:35.330996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:35.466041 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461990bc9adb4dcf9c815e8230469063', 'request_id': 'req-05357eca-3543-49e8-a71d-a64f7c7b6e4b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.526656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461990bc9adb4dcf9c815e8230469063', 'request_id': 'req-05357eca-3543-49e8-a71d-a64f7c7b6e4b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.534436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461990bc9adb4dcf9c815e8230469063', 'request_id': 'req-05357eca-3543-49e8-a71d-a64f7c7b6e4b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.551304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] The token cache is being invalidate because project 461990bc9adb4dcf9c815e8230469063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:35.551304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.554814 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.556700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1229/2457] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:35 2022] DELETE /identity/v3/projects/461990bc9adb4dcf9c815e8230469063 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:35.560517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:35.573148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c82536-294d-4100-9a2d-36f202873746 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:35.668837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:35.669746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:35.670179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:35.670634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] PATH_INFO: `/v3/users/91c3f809f7d5464d9021c0b00e513d5f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:35.671974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c3f809f7d5464d9021c0b00e513d5f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:35.674487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:35.691634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:35.906581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c3f809f7d5464d9021c0b00e513d5f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.978317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c3f809f7d5464d9021c0b00e513d5f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.978970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c3f809f7d5464d9021c0b00e513d5f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.992414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c3f809f7d5464d9021c0b00e513d5f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:35.999551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9b81a62-35b3-4bfc-83b9-82734a64084b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c3f809f7d5464d9021c0b00e513d5f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:36.012366 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1229/2458] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:35 2022] DELETE /identity/v3/users/91c3f809f7d5464d9021c0b00e513d5f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:36.493754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:36.495590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:36.519793 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05357eca-3543-49e8-a71d-a64f7c7b6e4b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:36.598174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:36.598882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:36.599508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:36.599886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] PATH_INFO: `/v3/projects/fa2e7edacbc7428f978047bd77ae1b67` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:36.602255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2e7edacbc7428f978047bd77ae1b67)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:36.604397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:36.716247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:36.727334 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2e7edacbc7428f978047bd77ae1b67', 'request_id': 'req-21bcf62a-f278-4543-8fa7-5a2826de0306'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:36.796756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2e7edacbc7428f978047bd77ae1b67', 'request_id': 'req-21bcf62a-f278-4543-8fa7-5a2826de0306'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:36.807643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2e7edacbc7428f978047bd77ae1b67', 'request_id': 'req-21bcf62a-f278-4543-8fa7-5a2826de0306'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:36.823138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] The token cache is being invalidate because project fa2e7edacbc7428f978047bd77ae1b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:36.823454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:36.824228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-21bcf62a-f278-4543-8fa7-5a2826de0306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:36.826390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1230/2459] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:36 2022] DELETE /identity/v3/projects/fa2e7edacbc7428f978047bd77ae1b67 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:36.830986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:36.831498 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:36.831599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:36.861816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:36.863373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:36.973814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1230/2460] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:36.975200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:36.976270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:36.976746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.013214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:37.014531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:37.024685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1231/2461] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:37.029025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.039867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf2d64-c9cc-4b10-81a8-d0f8004df2ad None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.044569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.045203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.045484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.045912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.046642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:40:37.046949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.048580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:37.063932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1231/2462] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:40:37.068352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.078412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b5c94-a3a8-4e8b-9bbf-8fa615352e15 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.083404 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.084076 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.086523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.086836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.087466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.089483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:37.211416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1232/2463] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:37 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:37.215625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.228761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d69bea-f092-460a-97ac-66dceafe9b75 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.297580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.298111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.298421 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.298692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.299321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.301256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:37.403799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1232/2464] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:37 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:37.407945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.425120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2adff-0b93-40d4-8e34-2e930e5f1930 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.431877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.432429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.432714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.432988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.433858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.435692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:37.454777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1233/2465] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:37.457065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.468615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6425dec-2a5a-4b1b-82a8-a0e44607f528 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.475069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70997f-40b6-440d-b5ed-eab7123e4e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.476252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70997f-40b6-440d-b5ed-eab7123e4e56 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.476545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70997f-40b6-440d-b5ed-eab7123e4e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.476832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70997f-40b6-440d-b5ed-eab7123e4e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.477480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70997f-40b6-440d-b5ed-eab7123e4e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.479683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70997f-40b6-440d-b5ed-eab7123e4e56 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:37.481435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:37.497737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1233/2466] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:37.499075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.511447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6711d-c1d8-4632-aba0-d0f14f0a36c2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.533574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.534911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.535429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.535884 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] PATH_INFO: `/v3/projects/853c3893f9a54ee497b50743d3c95e4c/users/9274c9890e404e6580f60664166d3ba5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.569738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=853c3893f9a54ee497b50743d3c95e4c, user_id=9274c9890e404e6580f60664166d3ba5, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.572495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:37.655123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1234/2467] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:37 2022] PUT /identity/v3/projects/853c3893f9a54ee497b50743d3c95e4c/users/9274c9890e404e6580f60664166d3ba5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:37.666500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.666788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.667063 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.700497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] MFA Rules not processed for user `9274c9890e404e6580f60664166d3ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:37.701796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:37.782763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1234/2468] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:37.787821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:37.865135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45896e6-7c55-4d6a-a222-c42de01dd518 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:37.871057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:37.871886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:37.872167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:37.872439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:37.873103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:37.875805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.170010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1235/2469] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:37 2022] POST /identity/v3/projects => generated 371 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:40:38.177906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:38.191162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19fa46-4536-477e-9a3f-64308f424e7f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:38.280694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:38.281468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.281831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.282196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.282966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:38.285350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.413694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1235/2470] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:38 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:40:38.417638 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:38.430618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce26783-3d3c-4429-9473-6efd3e422e64 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:38.438525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:38.439340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.439752 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.440176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.440948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:38.442680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.455354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1236/2471] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:38.459377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:38.470657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096d38d-f6c7-438a-a5cc-33ec216d160f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:38.475967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:38.476679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.476962 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.477236 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] PATH_INFO: `/v3/projects/b0921ed5d3344ea4a80378043777f12f/users/7bf519ca27924b7c9749e15d6d1b21fb/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.496050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0921ed5d3344ea4a80378043777f12f, user_id=7bf519ca27924b7c9749e15d6d1b21fb, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:38.499199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.572976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1236/2472] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:38 2022] PUT /identity/v3/projects/b0921ed5d3344ea4a80378043777f12f/users/7bf519ca27924b7c9749e15d6d1b21fb/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:38.577745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:38.590044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad4af5-2685-4edb-aee6-8448c6901990 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:38.641917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:38.667107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1315edd-7cd6-43cd-8381-cbe6457dae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:38.668387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1315edd-7cd6-43cd-8381-cbe6457dae37 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.668906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1315edd-7cd6-43cd-8381-cbe6457dae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.669414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1315edd-7cd6-43cd-8381-cbe6457dae37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.670218 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1315edd-7cd6-43cd-8381-cbe6457dae37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:38.672110 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1315edd-7cd6-43cd-8381-cbe6457dae37 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.692891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1237/2473] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:40:38.699801 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:38.712408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df513045-c6ec-4c34-9e1c-02284c8c0896 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:38.719450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:38.720132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.720553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.720840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] PATH_INFO: `/v3/projects/b0921ed5d3344ea4a80378043777f12f/users/7bf519ca27924b7c9749e15d6d1b21fb/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.722955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0921ed5d3344ea4a80378043777f12f, user_id=7bf519ca27924b7c9749e15d6d1b21fb, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:38.725933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.737384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1237/2474] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:38 2022] PUT /identity/v3/projects/b0921ed5d3344ea4a80378043777f12f/users/7bf519ca27924b7c9749e15d6d1b21fb/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:38.748047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.748601 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.748957 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.795442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] MFA Rules not processed for user `7bf519ca27924b7c9749e15d6d1b21fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:38.796786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:38.873590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1238/2475] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:38.885583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:38.888083 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:38.932803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:38.948380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb12db-dc67-4187-95e7-2a4e4da05f8c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:38.952241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232c4ff-a799-434e-a72b-cd0f5b9490da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:38.952850 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c4ff-a799-434e-a72b-cd0f5b9490da service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:38.953129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c4ff-a799-434e-a72b-cd0f5b9490da service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:38.953399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232c4ff-a799-434e-a72b-cd0f5b9490da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:38.963233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c4ff-a799-434e-a72b-cd0f5b9490da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:38.964808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232c4ff-a799-434e-a72b-cd0f5b9490da service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:38.977517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1238/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:40:39.926325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:39.988603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8375ac-1187-4452-b065-0c23b0683c33 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:39.993498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:39.994205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:39.994513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:39.994782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] PATH_INFO: `/v3/users/9274c9890e404e6580f60664166d3ba5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:39.995890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9274c9890e404e6580f60664166d3ba5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:39.997795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:40.186662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9274c9890e404e6580f60664166d3ba5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:40.187321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9274c9890e404e6580f60664166d3ba5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:40.198295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:40.199613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9274c9890e404e6580f60664166d3ba5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:40.206205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9274c9890e404e6580f60664166d3ba5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:40.246833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9274c9890e404e6580f60664166d3ba5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:40.255586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1239/2477] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:39 2022] DELETE /identity/v3/users/9274c9890e404e6580f60664166d3ba5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:40.260135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:40.260622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:40.260740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:40.281492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:40.283106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:40.343287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1239/2478] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:40.353468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:40.358908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:40.413557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908904da-5315-45fa-86a3-72bd820a2953 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:40.417783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:40.418439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:40.418718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:40.418993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:40.430165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:40.432204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:40.443554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1240/2479] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:40.911306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:40.912817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:40.926254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa047b-4a1f-4119-b5a1-67015314d16f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:40.932362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:40.933106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:40.933452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:40.933791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] PATH_INFO: `/v3/projects/853c3893f9a54ee497b50743d3c95e4c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:40.935012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=853c3893f9a54ee497b50743d3c95e4c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:40.937241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:41.056608 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '853c3893f9a54ee497b50743d3c95e4c', 'request_id': 'req-bbc71f92-b362-4272-833e-b1b21c82d8ba'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.155212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '853c3893f9a54ee497b50743d3c95e4c', 'request_id': 'req-bbc71f92-b362-4272-833e-b1b21c82d8ba'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.163096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '853c3893f9a54ee497b50743d3c95e4c', 'request_id': 'req-bbc71f92-b362-4272-833e-b1b21c82d8ba'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.175114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] The token cache is being invalidate because project 853c3893f9a54ee497b50743d3c95e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:41.175395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.176188 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.177876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1240/2480] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:40 2022] DELETE /identity/v3/projects/853c3893f9a54ee497b50743d3c95e4c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:41.182875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:41.195859 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2ec38-7e82-4beb-a283-da1494550b96 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:41.259842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:41.259842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:41.259842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:41.260137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] PATH_INFO: `/v3/users/7bf519ca27924b7c9749e15d6d1b21fb` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:41.261191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf519ca27924b7c9749e15d6d1b21fb)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:41.263205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:41.484127 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf519ca27924b7c9749e15d6d1b21fb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.484784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf519ca27924b7c9749e15d6d1b21fb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.497316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf519ca27924b7c9749e15d6d1b21fb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.504582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf519ca27924b7c9749e15d6d1b21fb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.605426 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8ef54cfe-d0b7-4a49-8b80-23d62814672c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf519ca27924b7c9749e15d6d1b21fb'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:41.616786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1241/2481] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:41 2022] DELETE /identity/v3/users/7bf519ca27924b7c9749e15d6d1b21fb => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:42.255254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:42.256862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:42.272937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71f92-b362-4272-833e-b1b21c82d8ba admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:42.344414 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:42.345143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:42.345577 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:42.345853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] PATH_INFO: `/v3/projects/b0921ed5d3344ea4a80378043777f12f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:42.347007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0921ed5d3344ea4a80378043777f12f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:42.348939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:42.357587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:43.357393 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0921ed5d3344ea4a80378043777f12f', 'request_id': 'req-361de1e9-b082-4b94-adeb-b28ecc567034'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:43.515202 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0921ed5d3344ea4a80378043777f12f', 'request_id': 'req-361de1e9-b082-4b94-adeb-b28ecc567034'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:43.522864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0921ed5d3344ea4a80378043777f12f', 'request_id': 'req-361de1e9-b082-4b94-adeb-b28ecc567034'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:43.534873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] The token cache is being invalidate because project b0921ed5d3344ea4a80378043777f12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:43.535148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:43.535801 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-361de1e9-b082-4b94-adeb-b28ecc567034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:43.537331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1241/2482] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:42 2022] DELETE /identity/v3/projects/b0921ed5d3344ea4a80378043777f12f => generated 0 bytes in 1283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:43.542507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:43.542991 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:43.543193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:43.554205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:43.568468 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:43.569916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:43.662010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1242/2483] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:43.663800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:43.664091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:43.664377 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:43.706195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:43.707657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:43.725785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1242/2484] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:43.727012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:43.750029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f0b79-f7d8-473f-897b-3f8108e1ed8a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:43.755747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:43.762649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:43.762649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:43.762649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:43.762649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:40:43.762649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:43.762649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:43.789333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1243/2485] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:40:43.794312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:43.808794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86e5a0-f23f-4ef2-bf63-600f0b90c83e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:43.834862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:43.835532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:43.835828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:43.836777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:43.837455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:43.839451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.001347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.008807 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1243/2486] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:43 2022] POST /identity/v3/projects => generated 363 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:40:44.013140 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9b81e-ea7c-4165-8dcf-e838e336bc81 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.075985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.076780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.077060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.077354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.077985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.079908 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.192791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1244/2487] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:44 2022] POST /identity/v3/users => generated 282 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:40:44.197681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.210482 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc55d2-8ceb-40b3-875e-b144b5ba07a9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.220186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.221163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.222303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.222600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.223467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.225312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.240088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1244/2488] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:44.243736 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.257899 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e604d-38af-427c-8ede-17605e8fadb0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.264247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa671d2-a49c-47a5-8e45-d16c952b0770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.264974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa671d2-a49c-47a5-8e45-d16c952b0770 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.265352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa671d2-a49c-47a5-8e45-d16c952b0770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.265834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa671d2-a49c-47a5-8e45-d16c952b0770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.266832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa671d2-a49c-47a5-8e45-d16c952b0770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.269136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa671d2-a49c-47a5-8e45-d16c952b0770 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.286394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1245/2489] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:40:44.291467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.302129 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df90a9b-6e05-4781-bfa9-d18bc3733bc3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.326766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.327460 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.327743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.328029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] PATH_INFO: `/v3/projects/ee34becd0572466abc29a5325e3a13da/users/9b9024603754480e811fa514c573b323/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.357294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee34becd0572466abc29a5325e3a13da, user_id=9b9024603754480e811fa514c573b323, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.360812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.448731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1245/2490] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:44 2022] PUT /identity/v3/projects/ee34becd0572466abc29a5325e3a13da/users/9b9024603754480e811fa514c573b323/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:40:44.459055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.459394 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.459611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.490642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] MFA Rules not processed for user `9b9024603754480e811fa514c573b323`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:44.492031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:44.563257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1246/2491] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:44.572452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.642675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3848c1b-7e63-4c25-ba21-df35b1d17b0d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.647483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.648095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.648378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.648651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.649322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.651368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.757930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1246/2492] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:40:44 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:44.762252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.773910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131d713-1533-4e08-8d99-acd8de367f84 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.837060 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.837680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.837961 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.838230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.838939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.840847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.930098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1247/2493] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:40:44 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:40:44.932918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.945476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3b9e7-8197-4c04-8cde-d9fccafa4374 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.953780 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.954549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.955210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.955210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:44.955721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:44.957327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:44.969770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1247/2494] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:40:44.974079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:44.984979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68ec9d-ecdf-455c-949f-7a33a89a3759 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:44.989503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:44.990157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:44.990477 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:44.990747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] PATH_INFO: `/v3/projects/7ad2aea92eb04bfe8ee19b232ad121cf/users/281e1dd8d58242e9a471f2e7731973e4/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:45.005498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad2aea92eb04bfe8ee19b232ad121cf, user_id=281e1dd8d58242e9a471f2e7731973e4, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:45.008891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:45.097598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1248/2495] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:44 2022] PUT /identity/v3/projects/7ad2aea92eb04bfe8ee19b232ad121cf/users/281e1dd8d58242e9a471f2e7731973e4/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:45.101532 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:45.108704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:45.112741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471b9a7-7d50-4891-866a-74677475ba96 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:45.168275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23084f-562c-4cda-8287-8fdda1bfe869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:45.168884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23084f-562c-4cda-8287-8fdda1bfe869 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:45.169161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23084f-562c-4cda-8287-8fdda1bfe869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:45.169429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23084f-562c-4cda-8287-8fdda1bfe869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:45.170059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23084f-562c-4cda-8287-8fdda1bfe869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:45.171727 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23084f-562c-4cda-8287-8fdda1bfe869 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:45.181273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1248/2496] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:40:45.185499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:45.195848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7eb79-fcbe-413f-a801-ee183f4a4c23 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:45.201102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:45.207984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:45.207984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:45.207984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] PATH_INFO: `/v3/projects/7ad2aea92eb04bfe8ee19b232ad121cf/users/281e1dd8d58242e9a471f2e7731973e4/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:45.207984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad2aea92eb04bfe8ee19b232ad121cf, user_id=281e1dd8d58242e9a471f2e7731973e4, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:45.207984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:45.219509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1249/2497] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:40:45 2022] PUT /identity/v3/projects/7ad2aea92eb04bfe8ee19b232ad121cf/users/281e1dd8d58242e9a471f2e7731973e4/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:45.231958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:45.232254 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:45.232531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:45.262168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] MFA Rules not processed for user `281e1dd8d58242e9a471f2e7731973e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:45.263380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:45.336168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1249/2498] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:45.348665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:45.351466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:45.411125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ad0e-25c6-4a10-8d4f-d60119fbc4e2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:45.414999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ab384-0408-4102-9ed6-a6122d81382d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:45.415653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ab384-0408-4102-9ed6-a6122d81382d service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:45.415936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ab384-0408-4102-9ed6-a6122d81382d service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:45.416175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ab384-0408-4102-9ed6-a6122d81382d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:45.425845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ab384-0408-4102-9ed6-a6122d81382d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:45.430255 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ab384-0408-4102-9ed6-a6122d81382d service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:45.438016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1250/2499] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:45.447274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:45.651273 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:45.661367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d5ec3-4762-4f5e-92fb-dd81f027d36c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:45.667416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:45.668307 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:45.668582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:45.668852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:45.728491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:45.729947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:45.740951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1250/2500] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:40:46.056640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:46.057153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:46.057153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:46.077199 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:46.078602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:46.152659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:46.161524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1251/2501] 10.222.0.13 () {58 vars in 967 bytes} [Sat Sep 3 01:40:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:40:46.174871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:46.176299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:46.220172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:46.254601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e74d9c-203e-467c-8697-349c4a84010d service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:46.260456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5ac44-1466-498f-9524-51cb8f6740ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:46.261066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ac44-1466-498f-9524-51cb8f6740ad service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:46.261345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ac44-1466-498f-9524-51cb8f6740ad service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:46.261615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ac44-1466-498f-9524-51cb8f6740ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:46.269817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5ac44-1466-498f-9524-51cb8f6740ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:46.271428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5ac44-1466-498f-9524-51cb8f6740ad service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:46.280568 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1251/2502] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:40:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:40:56.457764 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=21 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:56.531622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:57.529115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:57.634996 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077fb07f-04f8-4d71-948d-a696d4d4152e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:57.639641 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:57.640275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:57.640552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:57.641960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] PATH_INFO: `/v3/users/9b9024603754480e811fa514c573b323` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:57.641960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9024603754480e811fa514c573b323)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:57.643865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:57.837691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9024603754480e811fa514c573b323'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:57.838432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9024603754480e811fa514c573b323'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:57.859224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9024603754480e811fa514c573b323'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:57.873172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9024603754480e811fa514c573b323'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:57.926212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9024603754480e811fa514c573b323'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:57.933179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1252/2503] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:57 2022] DELETE /identity/v3/users/9b9024603754480e811fa514c573b323 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:57.937729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:57.938185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:57.938309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:57.997443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:40:57.998744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:58.064646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1252/2504] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:40:58.070433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:58.072510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:58.114544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:58.147673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480fcbb-0552-40dd-8e1f-a8e2ed40779c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:58.156722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:58.157356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:58.157645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:58.157920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:58.168803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:58.170284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:58.179865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1253/2505] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:40:58.640076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:58.641640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:58.664345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e1f1c-32ba-499a-b49d-9d57e6d8a6c0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:58.671244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:58.671988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:58.672337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:58.672690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] PATH_INFO: `/v3/projects/ee34becd0572466abc29a5325e3a13da` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:58.674100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee34becd0572466abc29a5325e3a13da)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:58.679028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:58.792618 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee34becd0572466abc29a5325e3a13da', 'request_id': 'req-74daf674-6475-4e89-97e8-fc62d37b4bfc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:58.841001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee34becd0572466abc29a5325e3a13da', 'request_id': 'req-74daf674-6475-4e89-97e8-fc62d37b4bfc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:58.847343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee34becd0572466abc29a5325e3a13da', 'request_id': 'req-74daf674-6475-4e89-97e8-fc62d37b4bfc'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:58.859462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] The token cache is being invalidate because project ee34becd0572466abc29a5325e3a13da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:40:58.859753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:58.860382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:58.864648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1253/2506] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:58 2022] DELETE /identity/v3/projects/ee34becd0572466abc29a5325e3a13da => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:58.865948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:58.876924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005715a-fdb2-42a3-a7df-7edab3af33e8 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:58.955500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:58.956132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:58.956418 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:58.956684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] PATH_INFO: `/v3/users/281e1dd8d58242e9a471f2e7731973e4` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:58.958373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] RBAC: Authorizing `identity:delete_user(user_id=281e1dd8d58242e9a471f2e7731973e4)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:58.960331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:40:59.135212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281e1dd8d58242e9a471f2e7731973e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:59.135818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281e1dd8d58242e9a471f2e7731973e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:59.148215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281e1dd8d58242e9a471f2e7731973e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:59.155373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281e1dd8d58242e9a471f2e7731973e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:59.268442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31114a51-67fb-4d0e-93a2-611bceb9e21c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281e1dd8d58242e9a471f2e7731973e4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:40:59.281780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1254/2507] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:40:58 2022] DELETE /identity/v3/users/281e1dd8d58242e9a471f2e7731973e4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:40:59.752046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:40:59.753612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:40:59.774418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daf674-6475-4e89-97e8-fc62d37b4bfc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:40:59.864309 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:40:59.880817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:40:59.882457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:40:59.882813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:40:59.883758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] PATH_INFO: `/v3/projects/7ad2aea92eb04bfe8ee19b232ad121cf` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:40:59.886703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad2aea92eb04bfe8ee19b232ad121cf)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:40:59.889991 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.015456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad2aea92eb04bfe8ee19b232ad121cf', 'request_id': 'req-e003cf5c-b609-48f3-9579-231dcf46a84f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:00.125368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad2aea92eb04bfe8ee19b232ad121cf', 'request_id': 'req-e003cf5c-b609-48f3-9579-231dcf46a84f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:00.131815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad2aea92eb04bfe8ee19b232ad121cf', 'request_id': 'req-e003cf5c-b609-48f3-9579-231dcf46a84f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:00.144456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] The token cache is being invalidate because project 7ad2aea92eb04bfe8ee19b232ad121cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:00.144749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:00.145511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e003cf5c-b609-48f3-9579-231dcf46a84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:00.153714 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1254/2508] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:40:59 2022] DELETE /identity/v3/projects/7ad2aea92eb04bfe8ee19b232ad121cf => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:00.158032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.158558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.158737 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.235708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:00.236950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:00.252450 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:00.307650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1255/2509] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:00.315812 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.316105 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.316380 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.338931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:00.340256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:00.351717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1255/2510] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:00.360729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:00.372387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81383fd4-d259-40b1-9321-3ab90caa2801 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:00.380160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:00.380796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.381095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.381352 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.382014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:00.382399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:00.388498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.407630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1256/2511] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:00.413552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:00.429994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593a3f7-e399-44d6-9521-a32835a66148 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:00.437360 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:00.438695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.438980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.439255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.439904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:00.441992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.574853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1256/2512] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:00 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:00.578610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:00.589505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298fd5-366d-4363-af86-a29fa2276eb0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:00.671465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:00.671918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.672205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.672479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.673144 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:00.675276 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.773019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1257/2513] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:00 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:00.778283 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:00.793457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5199f2-7d38-4b89-82a5-86f52243e693 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:00.804124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:00.812223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.812223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.812223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.812223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:00.812223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.845703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1257/2514] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:00.849682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:00.862832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabce9f-5453-484f-85b8-e484e8d2889c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:00.867875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af3cde-0e1f-496a-83a8-1573660ebf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:00.868708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3cde-0e1f-496a-83a8-1573660ebf50 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.869492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3cde-0e1f-496a-83a8-1573660ebf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.869492 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af3cde-0e1f-496a-83a8-1573660ebf50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.870084 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3cde-0e1f-496a-83a8-1573660ebf50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:00.871724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af3cde-0e1f-496a-83a8-1573660ebf50 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.883303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1258/2515] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:00.888070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:00.900145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd18038-e11a-4514-9810-125e1cfa7a1d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:00.906172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:00.906867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:00.907165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:00.907437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] PATH_INFO: `/v3/projects/4bc686c8bec044aeb24195fa5d1dda6e/users/280f75159a194a6dbeae19e92c6561d2/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:00.924354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc686c8bec044aeb24195fa5d1dda6e, user_id=280f75159a194a6dbeae19e92c6561d2, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:00.927219 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:00.989955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1258/2516] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:00 2022] PUT /identity/v3/projects/4bc686c8bec044aeb24195fa5d1dda6e/users/280f75159a194a6dbeae19e92c6561d2/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:01.000817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.001106 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.001408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.031431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] MFA Rules not processed for user `280f75159a194a6dbeae19e92c6561d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:01.032599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:01.109416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1259/2517] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:01.114280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.189741 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6129211-9ad9-4536-ab05-22b67ce19aab admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:01.194655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:01.195270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.195547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.195817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.197183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:01.199205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:01.299947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1259/2518] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:01 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:01.304130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.315786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdb299-3b50-4bfb-96f7-627ee5db2f25 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:01.328984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:01.388903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:01.389762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.390210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.390789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.391786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:01.394831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:01.523321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1260/2519] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:01 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:01.524705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.535485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f56ad6-bee9-4123-ae43-40b4578877c6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:01.540137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:01.540735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.541016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.541289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.542037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:01.543700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:01.545927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:01.555048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1260/2520] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:01.560032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.570368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98dae1-ebc2-4b63-bc05-11115b52adab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:01.575620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:01.576272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.576581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.576875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] PATH_INFO: `/v3/projects/78c6202963c74e649dcd7fce4e9592b9/users/0bdf244025094f86a0441fa715f22fb6/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.592121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c6202963c74e649dcd7fce4e9592b9, user_id=0bdf244025094f86a0441fa715f22fb6, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:01.594847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:01.709803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1261/2521] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:01 2022] PUT /identity/v3/projects/78c6202963c74e649dcd7fce4e9592b9/users/0bdf244025094f86a0441fa715f22fb6/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:01.715349 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.724745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40917688-9804-4479-997b-c3d30452f7da admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:01.800350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e73e-931b-41b0-bd3f-9c602c1c60b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:01.800984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e73e-931b-41b0-bd3f-9c602c1c60b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.801248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e73e-931b-41b0-bd3f-9c602c1c60b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.801551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e73e-931b-41b0-bd3f-9c602c1c60b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.802210 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e73e-931b-41b0-bd3f-9c602c1c60b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:01.804011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e73e-931b-41b0-bd3f-9c602c1c60b7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:01.817475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1261/2522] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:01.818769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.831759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b703dd5-b20f-4d76-988e-67f8da9c61c5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:01.837756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:01.839788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.839788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.839788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] PATH_INFO: `/v3/projects/78c6202963c74e649dcd7fce4e9592b9/users/0bdf244025094f86a0441fa715f22fb6/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.841421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c6202963c74e649dcd7fce4e9592b9, user_id=0bdf244025094f86a0441fa715f22fb6, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:01.844406 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:01.857261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1262/2523] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:01 2022] PUT /identity/v3/projects/78c6202963c74e649dcd7fce4e9592b9/users/0bdf244025094f86a0441fa715f22fb6/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:01.867737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d84381-70f1-4272-8318-10102d685906 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:01.868048 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d84381-70f1-4272-8318-10102d685906 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:01.868473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d84381-70f1-4272-8318-10102d685906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:01.897837 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c8d84381-70f1-4272-8318-10102d685906 None None] MFA Rules not processed for user `0bdf244025094f86a0441fa715f22fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:01.899090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c8d84381-70f1-4272-8318-10102d685906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:01.980552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1262/2524] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:01.989016 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:01.991520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:02.054052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cce85-19fa-485b-86bd-2cbfcf38d539 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:02.057765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec91c9-8d1a-45a8-8c1a-c7284e650ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:02.058303 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec91c9-8d1a-45a8-8c1a-c7284e650ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:02.058612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec91c9-8d1a-45a8-8c1a-c7284e650ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:02.058878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec91c9-8d1a-45a8-8c1a-c7284e650ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:02.069621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec91c9-8d1a-45a8-8c1a-c7284e650ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:02.071114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec91c9-8d1a-45a8-8c1a-c7284e650ea9 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:02.082777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1263/2525] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:02.154018 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d84381-70f1-4272-8318-10102d685906 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:02.226620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d84381-70f1-4272-8318-10102d685906 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:02.232857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:02.233651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:02.233937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:02.234239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] PATH_INFO: `/v3/users/280f75159a194a6dbeae19e92c6561d2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:02.235435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280f75159a194a6dbeae19e92c6561d2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:02.237391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:02.260469 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:02.526594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280f75159a194a6dbeae19e92c6561d2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:02.600292 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280f75159a194a6dbeae19e92c6561d2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:02.600925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280f75159a194a6dbeae19e92c6561d2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:02.615936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280f75159a194a6dbeae19e92c6561d2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:02.624006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280f75159a194a6dbeae19e92c6561d2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:02.630635 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1263/2526] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:02 2022] DELETE /identity/v3/users/280f75159a194a6dbeae19e92c6561d2 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:02.635841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:02.636312 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:02.636429 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:02.655962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:02.657182 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:02.738934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1264/2527] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:02.752128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:02.757796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:02.822424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520855e-7164-4b21-b98f-1978c995fd23 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:02.831767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:02.832760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:02.833359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:02.833929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:02.845308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:02.847682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:02.861208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1264/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:03.431629 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:03.433079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:03.446606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1fcf3-69e6-4483-ad77-60276d8d95b0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:03.452371 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:03.453030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:03.453315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:03.453583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] PATH_INFO: `/v3/projects/4bc686c8bec044aeb24195fa5d1dda6e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:03.454766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc686c8bec044aeb24195fa5d1dda6e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:03.457215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:03.469444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:03.582318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc686c8bec044aeb24195fa5d1dda6e', 'request_id': 'req-8a475e2b-b0a6-4043-91ff-2280638c5bbd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:03.662206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc686c8bec044aeb24195fa5d1dda6e', 'request_id': 'req-8a475e2b-b0a6-4043-91ff-2280638c5bbd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:03.671347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc686c8bec044aeb24195fa5d1dda6e', 'request_id': 'req-8a475e2b-b0a6-4043-91ff-2280638c5bbd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:03.684885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] The token cache is being invalidate because project 4bc686c8bec044aeb24195fa5d1dda6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:03.685130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:03.690105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:03.690105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1265/2529] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:03 2022] DELETE /identity/v3/projects/4bc686c8bec044aeb24195fa5d1dda6e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:03.691371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:03.703578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d888a6a-98b0-4e4d-9f4c-3c747d41978e service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:03.862133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:03.863284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:03.863823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:03.864316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] PATH_INFO: `/v3/users/0bdf244025094f86a0441fa715f22fb6` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:03.866266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bdf244025094f86a0441fa715f22fb6)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:03.870231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:04.120720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:04.123174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bdf244025094f86a0441fa715f22fb6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.228982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bdf244025094f86a0441fa715f22fb6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.229603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bdf244025094f86a0441fa715f22fb6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.243286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bdf244025094f86a0441fa715f22fb6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.250413 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bdf244025094f86a0441fa715f22fb6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.256479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1265/2530] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:03 2022] DELETE /identity/v3/users/0bdf244025094f86a0441fa715f22fb6 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:04.670966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:04.674300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:04.686751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475e2b-b0a6-4043-91ff-2280638c5bbd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:04.757661 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:04.758301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:04.758606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:04.758962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] PATH_INFO: `/v3/projects/78c6202963c74e649dcd7fce4e9592b9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:04.760134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c6202963c74e649dcd7fce4e9592b9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:04.762116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:04.864605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:04.866043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:04.899122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c6202963c74e649dcd7fce4e9592b9', 'request_id': 'req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.958135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18879189-c4de-410e-a8f3-5099864052a0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:04.971583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c6202963c74e649dcd7fce4e9592b9', 'request_id': 'req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.979416 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c6202963c74e649dcd7fce4e9592b9', 'request_id': 'req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.990660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] The token cache is being invalidate because project 78c6202963c74e649dcd7fce4e9592b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:04.990948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.991605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-68563bd2-f223-44ed-a2d8-8d5fb09f7cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:04.993138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1266/2531] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:04 2022] DELETE /identity/v3/projects/78c6202963c74e649dcd7fce4e9592b9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:04.998509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:04.998874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:04.999160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.018144 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.018887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.019171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.019389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:05.019534 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.020612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:05.021182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:05.033056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.034960 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db99d4-5561-4831-b425-5b37f9dd2d00 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.043637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:05.046773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1266/2532] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:05.092706 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1267/2533] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:05.100312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.100636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.100914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.120777 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:05.122020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:05.130388 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1267/2534] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:05.135183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.144703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8782ec-3b19-4677-bb08-ad97bbc35d95 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.149934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.150560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.150868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.151138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.151779 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:05.152079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.153668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.165794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1268/2535] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:41:05.170316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.179705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78753e-8414-48e0-8f32-423d7ed58e21 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.184944 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.186694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.186694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.186694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.186966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.188904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.291453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1268/2536] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:05 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:05.297473 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.309043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580d0c8-e2bd-4ca0-a2c9-dbfc7b81e13c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.382313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.382971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.383254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.383524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.384251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.386316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.549095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1269/2537] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:05 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:05.553458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.560101 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:05.563785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6830cb-a711-4046-be72-115ed7d54f96 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.568367 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.568961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.569242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.569512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.570196 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.572355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.582212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1269/2538] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:05.586224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.596781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6a9cc-18a5-4c23-ba59-acf18c2baae4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.601534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bb218-cbd4-4640-ab84-d2669105aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.602134 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb218-cbd4-4640-ab84-d2669105aee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.602582 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb218-cbd4-4640-ab84-d2669105aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.602816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bb218-cbd4-4640-ab84-d2669105aee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.603337 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bb218-cbd4-4640-ab84-d2669105aee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.604937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bb218-cbd4-4640-ab84-d2669105aee4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.618931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1270/2539] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:05.620164 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.641276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9585b22-23ff-4705-8433-0163ed337c90 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.646114 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.646888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.647172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.647611 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] PATH_INFO: `/v3/projects/1b551c39979e47ab98442ab4475f1980/users/60fe8997b740430aac275c4a48237420/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.664164 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b551c39979e47ab98442ab4475f1980, user_id=60fe8997b740430aac275c4a48237420, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.667930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.726025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1270/2540] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:05 2022] PUT /identity/v3/projects/1b551c39979e47ab98442ab4475f1980/users/60fe8997b740430aac275c4a48237420/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:05.751191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.751707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.751808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.779918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] MFA Rules not processed for user `60fe8997b740430aac275c4a48237420`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:05.781118 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:05.851738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1271/2541] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:05.856696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:05.918522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d3911-2468-413d-bfb7-9125b2ec7397 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:05.923286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:05.923881 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:05.924161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:05.924447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:05.925086 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:05.927030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:05.995537 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1271/2542] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:05 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:06.009068 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.019806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dec332-a644-4e83-9977-024b771e4f07 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.043120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:06.083030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.083564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.083886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.084153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.084776 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.086721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.182964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1272/2543] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:06 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:06.186955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.197143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ca088-4a2f-4d4b-8fa5-ae6d099a6bb6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.202804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.203401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.203682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.203952 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.204592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.206189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.217199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1272/2544] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:06.221264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.232108 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9edd39-e2fe-43d3-be4a-d4938143eae4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.236806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.237459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.237745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.238177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] PATH_INFO: `/v3/projects/f9a1b43e276047d7a518d0de73cb373c/users/8e420bb8ed874032beedb870520b422b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.252902 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a1b43e276047d7a518d0de73cb373c, user_id=8e420bb8ed874032beedb870520b422b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.255714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.326209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1273/2545] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:06 2022] PUT /identity/v3/projects/f9a1b43e276047d7a518d0de73cb373c/users/8e420bb8ed874032beedb870520b422b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:06.332948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.343543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c80484-49bf-45bb-b8ff-b2f437dac251 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.398330 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9f8f3-af0e-4d5d-ae70-ee109525a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.398956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9f8f3-af0e-4d5d-ae70-ee109525a827 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.399238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9f8f3-af0e-4d5d-ae70-ee109525a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.400063 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9f8f3-af0e-4d5d-ae70-ee109525a827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.400700 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9f8f3-af0e-4d5d-ae70-ee109525a827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.402301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9f8f3-af0e-4d5d-ae70-ee109525a827 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.412199 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1273/2546] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:06.416569 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.426773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460758-1f43-4c40-9232-909623974fb9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.431454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.432092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.432375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.432647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] PATH_INFO: `/v3/projects/f9a1b43e276047d7a518d0de73cb373c/users/8e420bb8ed874032beedb870520b422b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.434595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a1b43e276047d7a518d0de73cb373c, user_id=8e420bb8ed874032beedb870520b422b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.437231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.447780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1274/2547] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:06 2022] PUT /identity/v3/projects/f9a1b43e276047d7a518d0de73cb373c/users/8e420bb8ed874032beedb870520b422b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:06.457696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.457988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.458270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.487425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] MFA Rules not processed for user `8e420bb8ed874032beedb870520b422b`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:06.488638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:06.516899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:06.556165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1274/2548] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:06.569985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.572306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:06.629728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4dfa0-5470-4c82-bb2b-f149742a1c3f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.633325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.634538 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.634821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.635090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.644529 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.645978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.656292 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1275/2549] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:06.777853 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.780187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:06.799848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:06.801510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:06.817358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c38acab-8f2e-4272-84c9-f10cec7e27a7 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.823364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.824056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.824340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.824576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.862992 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15476c-eb6f-4eee-bee5-8a37f82581ff None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:06.868561 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:06.869429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:06.869711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:06.869983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:06.871840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:06.896967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.898615 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.916043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1276/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:06.942903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:06.944371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60687137-5aa6-4f23-9c3f-64cd1f688fc7 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:06.957508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1275/2551] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:07.386834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:07.401114 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665c3bb-12eb-4623-b5b6-403eaeff16a8 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:07.403619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:07.404361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:07.404963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:07.405088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] PATH_INFO: `/v3/users/60fe8997b740430aac275c4a48237420` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:07.406269 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=60fe8997b740430aac275c4a48237420)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:07.408261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:07.415741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:07.562935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60fe8997b740430aac275c4a48237420'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:07.563946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60fe8997b740430aac275c4a48237420'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:07.577179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60fe8997b740430aac275c4a48237420'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:07.584977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60fe8997b740430aac275c4a48237420'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:07.655997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60fe8997b740430aac275c4a48237420'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:07.662678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1277/2552] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:07 2022] DELETE /identity/v3/users/60fe8997b740430aac275c4a48237420 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:07.668019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:07.668308 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:07.668587 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:07.688214 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:07.689437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:07.748717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1276/2553] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:07.758505 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:07.760157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:07.821940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8f393-d043-4da9-b185-cb486ca6de7f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:07.825885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:07.826533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:07.826818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:07.827089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:07.837887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:07.840846 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:07.849651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1278/2554] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:08.272658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:08.275531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:08.313950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c552e8-d699-4c65-b356-3a382c3fab87 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:08.318903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:08.319557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:08.319840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:08.320117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] PATH_INFO: `/v3/projects/1b551c39979e47ab98442ab4475f1980` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:08.321434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b551c39979e47ab98442ab4475f1980)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:08.323443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:08.334905 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:08.453314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b551c39979e47ab98442ab4475f1980', 'request_id': 'req-c403027f-2c0a-45ac-8d80-3308732129a3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.532013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b551c39979e47ab98442ab4475f1980', 'request_id': 'req-c403027f-2c0a-45ac-8d80-3308732129a3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.538778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b551c39979e47ab98442ab4475f1980', 'request_id': 'req-c403027f-2c0a-45ac-8d80-3308732129a3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.551022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] The token cache is being invalidate because project 1b551c39979e47ab98442ab4475f1980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:08.551310 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.552096 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.553688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1277/2555] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:08 2022] DELETE /identity/v3/projects/1b551c39979e47ab98442ab4475f1980 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:08.557769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:08.568570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64989fd0-f120-47b5-ad3b-9a09752fb0c2 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:08.659128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:08.660328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:08.660784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:08.670372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] PATH_INFO: `/v3/users/8e420bb8ed874032beedb870520b422b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:08.670372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e420bb8ed874032beedb870520b422b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:08.670372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:08.841389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:08.856037 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e420bb8ed874032beedb870520b422b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.856729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e420bb8ed874032beedb870520b422b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.869665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e420bb8ed874032beedb870520b422b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.876417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e420bb8ed874032beedb870520b422b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.943325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fba64599-9cc5-45a5-8506-668b15718dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e420bb8ed874032beedb870520b422b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:08.951332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1279/2556] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:08 2022] DELETE /identity/v3/users/8e420bb8ed874032beedb870520b422b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:09.385987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:09.388191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:09.406432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403027f-2c0a-45ac-8d80-3308732129a3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:09.475340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:09.475896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:09.476179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:09.476451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] PATH_INFO: `/v3/projects/f9a1b43e276047d7a518d0de73cb373c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:09.477593 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a1b43e276047d7a518d0de73cb373c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:09.479542 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:09.597899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a1b43e276047d7a518d0de73cb373c', 'request_id': 'req-4ba79465-9628-4fce-81a2-adb67075de42'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:09.669177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a1b43e276047d7a518d0de73cb373c', 'request_id': 'req-4ba79465-9628-4fce-81a2-adb67075de42'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:09.675551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a1b43e276047d7a518d0de73cb373c', 'request_id': 'req-4ba79465-9628-4fce-81a2-adb67075de42'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:09.688051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] The token cache is being invalidate because project f9a1b43e276047d7a518d0de73cb373c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:09.688340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:09.688987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4ba79465-9628-4fce-81a2-adb67075de42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:09.690511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1278/2557] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:09 2022] DELETE /identity/v3/projects/f9a1b43e276047d7a518d0de73cb373c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:09.695335 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:09.695616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:09.695886 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:09.717528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:09.718805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:09.780701 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1280/2558] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:09.788619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:09.788912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:09.789171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:09.810381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:09.811122 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:09.819142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1279/2559] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:09.827929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:09.834579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998af1f-8866-4cb4-8d06-a21e2cbc313e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:09.839395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:09.840051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:09.840478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:09.841311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:09.841415 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:09.841680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:09.843311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:09.855464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1281/2560] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:09.859784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:09.870258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f0cb-f20d-4143-82de-0f0aa4feffd2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:09.875082 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:09.875663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:09.875965 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:09.876275 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:09.876917 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:09.878925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:09.996120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:10.006901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1280/2561] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:09 2022] POST /identity/v3/projects => generated 357 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:10.009488 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.022457 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a48c4c-3c07-46bd-b1a7-ce8b9ba4a656 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.101937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.103346 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.103613 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.103967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.104639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.106648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.195587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1282/2562] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:10 2022] POST /identity/v3/users => generated 279 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:10.199916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.214974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6fb1e-0508-47af-8fe5-12d0e4a97bd5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.223379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.224473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.242565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.242565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.242565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.242565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.242565 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1281/2563] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:10.245785 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.257000 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1590-5cde-4e97-a4e2-4f2bd1f99a84 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.263522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8c772-1bc6-428d-a5de-a676f494ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.267236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8c772-1bc6-428d-a5de-a676f494ff28 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.267236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8c772-1bc6-428d-a5de-a676f494ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.267236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8c772-1bc6-428d-a5de-a676f494ff28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.267236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8c772-1bc6-428d-a5de-a676f494ff28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.267236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8c772-1bc6-428d-a5de-a676f494ff28 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.279306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1283/2564] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:10.283986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.296789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac667-5c7d-4605-9e99-563034607a64 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.311733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.311733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.311733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.311733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] PATH_INFO: `/v3/projects/f611138d9b11480d8d40872ea91bb474/users/bbf9fe6c249544c49dec831e370f3d62/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.325300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f611138d9b11480d8d40872ea91bb474, user_id=bbf9fe6c249544c49dec831e370f3d62, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.326529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.441220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1282/2565] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:10 2022] PUT /identity/v3/projects/f611138d9b11480d8d40872ea91bb474/users/bbf9fe6c249544c49dec831e370f3d62/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:10.512157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.512495 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.512790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.547849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:10.550943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] MFA Rules not processed for user `bbf9fe6c249544c49dec831e370f3d62`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:10.552163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:10.620489 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1284/2566] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:10.626088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.627912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:10.661670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.663148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:10.705296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ac00d-a615-417c-87af-67fe21a4fc4d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.710118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.711042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.711398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.712334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] PATH_INFO: `/v3/users/2b9ab1983bb84318b509878b930e9402` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.713582 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9ab1983bb84318b509878b930e9402)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.715841 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.735194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5c9bc-0310-4315-97a8-7aadd561d557 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.738839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.739482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.740059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.740342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.750324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.751951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.761536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1285/2567] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:10.778029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:10.793015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8232c41-2e74-43b8-b5b2-516e20b56978 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:10.794762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:10.795499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:10.795783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:10.796056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] PATH_INFO: `/v3/users/bbf9fe6c249544c49dec831e370f3d62` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:10.797322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf9fe6c249544c49dec831e370f3d62)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:10.799528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:10.900066 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9ab1983bb84318b509878b930e9402'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:10.954591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9ab1983bb84318b509878b930e9402'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:10.955261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9ab1983bb84318b509878b930e9402'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:10.968132 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9ab1983bb84318b509878b930e9402'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:10.975228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b11bfae-a922-4a32-b770-7b851f862fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9ab1983bb84318b509878b930e9402'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:10.982089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1283/2568] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:10 2022] DELETE /identity/v3/users/2b9ab1983bb84318b509878b930e9402 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:10.989211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf9fe6c249544c49dec831e370f3d62'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:10.990112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf9fe6c249544c49dec831e370f3d62'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.003483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf9fe6c249544c49dec831e370f3d62'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.010600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf9fe6c249544c49dec831e370f3d62'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.074539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf9fe6c249544c49dec831e370f3d62'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.081253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1286/2569] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:10 2022] DELETE /identity/v3/users/bbf9fe6c249544c49dec831e370f3d62 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:11.085755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.086038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.086306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.106802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:11.108021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:11.150669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:11.169072 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1284/2570] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:11.178785 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.180229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:11.203546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:11.232806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.232806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:11.248735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18a873-9e01-4f34-829d-dc9e872e1695 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.256181 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:11.260747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.260747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.260747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] PATH_INFO: `/v3/projects/02813cf0683749ee87b039b28f1aeebb` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.260747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=02813cf0683749ee87b039b28f1aeebb)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:11.262489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:11.266274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f474cb0-912a-4b0d-8483-a0ba82a2c503 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.270263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ad556-6f36-4f36-9c65-57f4b666faab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:11.270921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ad556-6f36-4f36-9c65-57f4b666faab service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.271204 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ad556-6f36-4f36-9c65-57f4b666faab service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.271475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ad556-6f36-4f36-9c65-57f4b666faab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.281500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ad556-6f36-4f36-9c65-57f4b666faab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:11.282971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ad556-6f36-4f36-9c65-57f4b666faab service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:11.293183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1287/2571] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:11.378768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02813cf0683749ee87b039b28f1aeebb', 'request_id': 'req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.440166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02813cf0683749ee87b039b28f1aeebb', 'request_id': 'req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.446012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02813cf0683749ee87b039b28f1aeebb', 'request_id': 'req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.461995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] The token cache is being invalidate because project 02813cf0683749ee87b039b28f1aeebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:11.461995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.461995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d6b298f-e0a2-4c61-9434-0ae0d9aed92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.461995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1285/2572] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:11 2022] DELETE /identity/v3/projects/02813cf0683749ee87b039b28f1aeebb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:11.467350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.467689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.467959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.487672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:11.488910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:11.555053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1288/2573] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:11.563517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.567201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.567201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.585391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:11.586708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:11.595355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1286/2574] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:11.599563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.610313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236493ac-5aff-4f54-88d4-c1bdf261374d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.615377 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:11.615973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.616249 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.616516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.617158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:11.617462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:11.619205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:11.630861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1289/2575] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:41:11.636112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.646089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d2805-ff26-45ff-8fe0-8d1fc0944ac9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.651847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:11.652451 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.652736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.653010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.653655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:11.655699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:11.716998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.718830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:11.733621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75319e-5b3f-4728-9687-4bc3e8889429 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.739541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:11.740264 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.740583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.740790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] PATH_INFO: `/v3/projects/f611138d9b11480d8d40872ea91bb474` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.741997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f611138d9b11480d8d40872ea91bb474)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:11.753206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:11.753653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1287/2576] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:11 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:11.753653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.815078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e0d99-0854-46c5-88a7-5cf98f416937 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.819886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:11.821624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.821624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.821987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:11.822461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:11.824418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:11.886963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f611138d9b11480d8d40872ea91bb474', 'request_id': 'req-f29421f6-d2f2-4b6b-af57-0150f600252d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.931466 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1288/2577] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:11 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:11.935449 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:11.945169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca2ac-6a35-4852-a41b-07a38066f41a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:11.953953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f611138d9b11480d8d40872ea91bb474', 'request_id': 'req-f29421f6-d2f2-4b6b-af57-0150f600252d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.964063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f611138d9b11480d8d40872ea91bb474', 'request_id': 'req-f29421f6-d2f2-4b6b-af57-0150f600252d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.972970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] The token cache is being invalidate because project f611138d9b11480d8d40872ea91bb474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:11.973148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:11.973282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.973885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f29421f6-d2f2-4b6b-af57-0150f600252d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:11.975595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1290/2578] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:11 2022] DELETE /identity/v3/projects/f611138d9b11480d8d40872ea91bb474 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:11.980633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927358b1-101e-48cf-a3eb-1c9c5febf2da None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:11.980922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927358b1-101e-48cf-a3eb-1c9c5febf2da None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:11.981193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927358b1-101e-48cf-a3eb-1c9c5febf2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.001183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-927358b1-101e-48cf-a3eb-1c9c5febf2da None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:12.003995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-927358b1-101e-48cf-a3eb-1c9c5febf2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:12.005751 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.006398 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.006681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.006950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.007590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.010106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.011023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1291/2579] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:12.018692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.018977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.019251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.023290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1289/2580] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:12.025207 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.027805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:12.027968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:12.038795 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:12.040191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:12.042872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d270b57-5edf-455b-825e-c3293d3cdcc8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.045168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.045774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.046079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.046382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.047017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.048278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1292/2581] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:12.048620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.052390 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.063070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1290/2582] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:12.067172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916093ea-1360-4353-be3f-8a31738fd275 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.067345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.067607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.068237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.068525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.068800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.069452 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:12.069760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.072147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.074042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220d47d-d3b0-433a-b083-ce24fd69a336 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.078926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd75a1-2950-45d9-8c8d-266273ddb5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.079581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd75a1-2950-45d9-8c8d-266273ddb5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.079868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd75a1-2950-45d9-8c8d-266273ddb5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.080141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd75a1-2950-45d9-8c8d-266273ddb5fe admin admin] PATH_INFO: `/v3/projects/31be2034a71241d99b13017142fc4297/users/67581bdc0c8d4344ae33cc0b974dc401/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.088688 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1293/2583] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:41:12.089715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.095120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd75a1-2950-45d9-8c8d-266273ddb5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=31be2034a71241d99b13017142fc4297, user_id=67581bdc0c8d4344ae33cc0b974dc401, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.097795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd75a1-2950-45d9-8c8d-266273ddb5fe admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.099244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cea90d-9974-45d6-bfa8-b82859b10cd2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.104125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.104726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.105002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.105273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.105893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.107858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.172075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1291/2584] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:12 2022] PUT /identity/v3/projects/31be2034a71241d99b13017142fc4297/users/67581bdc0c8d4344ae33cc0b974dc401/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:12.183675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.183961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.184231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.206020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1294/2585] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:12.207129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.214010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] MFA Rules not processed for user `67581bdc0c8d4344ae33cc0b974dc401`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:12.215306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:12.270598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba18d58-5f06-4d67-a267-6d31e20dcc84 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.275684 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.276293 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.276571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.276841 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.277467 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.278533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1292/2586] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:12.284768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.289744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.291576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:12.365455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1295/2587] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:12.370412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.377352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:12.381319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391391e-5c87-401a-b17c-f13cef5e867e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.386183 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.386813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.387153 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.387424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.388116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.389788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.399900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1296/2588] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:12.403474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8890676-d027-4e8f-a547-2c59c408640c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.403894 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.409077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.409953 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.410235 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.410560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.413632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c3eef-6ab2-4bff-a4fe-529bdb589016 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.418827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17697d4-389b-4e5c-a8a0-b6d21ccb9364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.419408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17697d4-389b-4e5c-a8a0-b6d21ccb9364 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.419536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.419693 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17697d4-389b-4e5c-a8a0-b6d21ccb9364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.420019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17697d4-389b-4e5c-a8a0-b6d21ccb9364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.420651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17697d4-389b-4e5c-a8a0-b6d21ccb9364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.421051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.422201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17697d4-389b-4e5c-a8a0-b6d21ccb9364 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.431408 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1293/2589] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:12.433588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1297/2590] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:12.437843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.448217 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb167-7622-4aa2-9478-cf4044b8ba49 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.457237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.457237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.457237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.457237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] PATH_INFO: `/v3/projects/09010bacbc1c46228168164dd1cda48a/users/b9c1de6281aa4bfbb58d489315ae1938/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.468793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09010bacbc1c46228168164dd1cda48a, user_id=b9c1de6281aa4bfbb58d489315ae1938, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.471520 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.554864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1294/2591] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:12 2022] PUT /identity/v3/projects/09010bacbc1c46228168164dd1cda48a/users/b9c1de6281aa4bfbb58d489315ae1938/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:12.565223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.565555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.565784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.595015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] MFA Rules not processed for user `b9c1de6281aa4bfbb58d489315ae1938`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:12.596250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:12.661057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1298/2592] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:12.665867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.728362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c0d6-6bb8-4713-901b-21fde912972c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.732945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.733546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.733821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.734089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.734743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.736632 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:12.813801 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1295/2593] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:12.817799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:12.829811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c8d3a-2ecc-4028-8fe2-4e3cccc688d2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:12.846407 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:12.887363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:12.887978 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:12.888257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:12.888527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:12.889215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:12.891176 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.012852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1299/2594] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:12 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:13.017020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.028008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233ebf4-520d-438a-a362-d74c59e6e347 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.032546 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.033143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.033422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.033691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.034601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.036215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.048804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1296/2595] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:13.053129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.063257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f913bb-6a16-4663-bfe7-031c0f0ff562 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.067918 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa47a66-456a-40d0-af00-6c5a75980c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.068521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa47a66-456a-40d0-af00-6c5a75980c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.068803 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa47a66-456a-40d0-af00-6c5a75980c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.069072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa47a66-456a-40d0-af00-6c5a75980c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.069712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa47a66-456a-40d0-af00-6c5a75980c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.071916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa47a66-456a-40d0-af00-6c5a75980c4b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.083417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1300/2596] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:13.088649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.099265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0406a-89a3-4bfb-ab0f-c640baff2f01 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.104655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.104945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.105221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.105487 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] PATH_INFO: `/v3/projects/29ec827eb3c24625a21801da49f1523a/users/289da527111d4166813b3aece74c7699/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.124768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ec827eb3c24625a21801da49f1523a, user_id=289da527111d4166813b3aece74c7699, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.124768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.200423 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1297/2597] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:13 2022] PUT /identity/v3/projects/29ec827eb3c24625a21801da49f1523a/users/289da527111d4166813b3aece74c7699/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:13.211986 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.212361 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.212710 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.241656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] MFA Rules not processed for user `289da527111d4166813b3aece74c7699`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:13.242974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:13.308313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1301/2598] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:13.312921 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.375146 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29805e-bee5-4dde-afcd-51c7dccf5d9f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.379802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.380399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.380680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.380948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.381614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.384106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.393284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:13.469710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1298/2599] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:13 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:13.473839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.484678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a499a6-1af5-471f-9e47-0e334ca12f68 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.539955 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.540553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.540828 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.541094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.541726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.543645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.629311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1302/2600] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:13 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:13.633506 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.644541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f252467-49c0-4d52-9a29-4f24fdbc5430 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.649724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.650314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.650621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.653015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.653015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.653646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.664221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1299/2601] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:13.668772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.678768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35190d-b3ff-42a9-b9cf-64b3fd4d8e37 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.683449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.684124 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.684366 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.684633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] PATH_INFO: `/v3/projects/81c5054d6c264244aa6f077707eb580c/users/9b48d0d60ba84bd8bc3e7a5b82ab7ff1/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.699253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c5054d6c264244aa6f077707eb580c, user_id=9b48d0d60ba84bd8bc3e7a5b82ab7ff1, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.701888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.704157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:13.851369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1303/2602] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:13 2022] PUT /identity/v3/projects/81c5054d6c264244aa6f077707eb580c/users/9b48d0d60ba84bd8bc3e7a5b82ab7ff1/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:13.857137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.867194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ea6a4-22f5-4a67-8354-c9b5362ad229 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.924174 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a5340-a9d1-4468-9a84-512754152c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.924756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5340-a9d1-4468-9a84-512754152c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.925030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5340-a9d1-4468-9a84-512754152c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.925293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5340-a9d1-4468-9a84-512754152c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.926317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a5340-a9d1-4468-9a84-512754152c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.927557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a5340-a9d1-4468-9a84-512754152c52 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.938020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1300/2603] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:13.942235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:13.953010 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dca6ef-8ce6-4f12-8272-cb0e234c4870 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:13.957573 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:13.958238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.958552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.958830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] PATH_INFO: `/v3/projects/81c5054d6c264244aa6f077707eb580c/users/9b48d0d60ba84bd8bc3e7a5b82ab7ff1/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:13.960762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c5054d6c264244aa6f077707eb580c, user_id=9b48d0d60ba84bd8bc3e7a5b82ab7ff1, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:13.963453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:13.973642 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1304/2604] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:13 2022] PUT /identity/v3/projects/81c5054d6c264244aa6f077707eb580c/users/9b48d0d60ba84bd8bc3e7a5b82ab7ff1/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:13.984623 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:13.984913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:13.985187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:14.013354 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] MFA Rules not processed for user `9b48d0d60ba84bd8bc3e7a5b82ab7ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:14.014580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:14.082014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1301/2605] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:14.094472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:14.097105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:14.161399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77a644-0297-40af-965e-d09efff6a2cb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:14.166003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166fd98-8da0-46a5-a7e7-11aecc3929ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:14.166963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166fd98-8da0-46a5-a7e7-11aecc3929ca service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:14.167246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166fd98-8da0-46a5-a7e7-11aecc3929ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:14.167512 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166fd98-8da0-46a5-a7e7-11aecc3929ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:14.229272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166fd98-8da0-46a5-a7e7-11aecc3929ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:14.230741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166fd98-8da0-46a5-a7e7-11aecc3929ca service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:14.239788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1305/2606] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:15.653831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:15.712384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d58cb4-1e79-4764-9df6-9748e048122e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:15.716906 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:15.717541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:15.717813 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:15.718088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] PATH_INFO: `/v3/users/67581bdc0c8d4344ae33cc0b974dc401` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:15.719231 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67581bdc0c8d4344ae33cc0b974dc401)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:15.721167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:15.722669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:15.928252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67581bdc0c8d4344ae33cc0b974dc401'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:15.977325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67581bdc0c8d4344ae33cc0b974dc401'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:15.977971 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67581bdc0c8d4344ae33cc0b974dc401'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:15.990566 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67581bdc0c8d4344ae33cc0b974dc401'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:15.998057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67581bdc0c8d4344ae33cc0b974dc401'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:16.004022 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1302/2607] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:15 2022] DELETE /identity/v3/users/67581bdc0c8d4344ae33cc0b974dc401 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:16.009151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:16.009656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:16.009656 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:16.029062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:16.030295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:16.090046 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1306/2608] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:16.100180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:16.101630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:16.172555 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1669c3d-c13d-4208-b66b-f4e55b1c8fa7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:16.178683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:16.179674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:16.180159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:16.181250 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:16.192028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:16.193427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:16.203911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1303/2609] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:16.236874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:16.238294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:16.282455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:16.301021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a1ecc-5b0c-454b-a6b6-9ea00f6b1622 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:16.304577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:16.305771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:16.306087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:16.306514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:16.308373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:16.369892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:16.371375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a520a-4556-4cf9-9f4d-e26461d9a1d0 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:16.380324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1307/2610] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:16.642642 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:16.644318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:16.660739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a11ecf-a0b1-45df-93af-2750cd1174e9 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:16.667268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:16.667550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:16.667844 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:16.668212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] PATH_INFO: `/v3/projects/31be2034a71241d99b13017142fc4297` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:16.669260 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31be2034a71241d99b13017142fc4297)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:16.671239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:16.808344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31be2034a71241d99b13017142fc4297', 'request_id': 'req-bb1a3d87-d794-48b7-874e-ec10298c92b3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:16.885668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31be2034a71241d99b13017142fc4297', 'request_id': 'req-bb1a3d87-d794-48b7-874e-ec10298c92b3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:16.891755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31be2034a71241d99b13017142fc4297', 'request_id': 'req-bb1a3d87-d794-48b7-874e-ec10298c92b3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:16.903474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] The token cache is being invalidate because project 31be2034a71241d99b13017142fc4297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:16.903758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:16.904397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb1a3d87-d794-48b7-874e-ec10298c92b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:16.905874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1304/2611] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:16 2022] DELETE /identity/v3/projects/31be2034a71241d99b13017142fc4297 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:16.911056 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:16.911365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:16.911639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:16.931263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:16.931263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:16.993310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1308/2612] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:17.001059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.001344 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.001620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.010763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:17.021755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:17.023016 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:17.031719 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1305/2613] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:17.036210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.046069 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f7b62-fb3c-4584-8d89-f12a95637fd8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.050723 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.051322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.051633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.051907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.052550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:17.052851 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.054479 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.066284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1309/2614] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:41:17.070637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.080264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304440d-6243-4c5f-8fa3-e60605dfba35 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.085055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.085653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.085933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.086201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.087471 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.089373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.173447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1306/2615] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:17 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:17.178265 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.189778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abd080-7dcc-44a8-8fe3-7bb22e33fa28 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.249480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.250236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.250550 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.250820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.251461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.253424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.355288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1310/2616] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:17 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:17.359369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.369938 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32048b62-fbd4-43df-b0c7-83ceb6ebe0d9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.374951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.375553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.375839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.376148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.376793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.378484 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.388504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1307/2617] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:17.392579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.403562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9d5d2-0a38-4a9a-a6e3-2c23693d5a66 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.408243 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de4b05-2b80-48fc-b504-691dbc46465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.408838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4b05-2b80-48fc-b504-691dbc46465b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.409121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4b05-2b80-48fc-b504-691dbc46465b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.409650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de4b05-2b80-48fc-b504-691dbc46465b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.410328 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de4b05-2b80-48fc-b504-691dbc46465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.412858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de4b05-2b80-48fc-b504-691dbc46465b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.413238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:17.424098 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1311/2618] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:17.428404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.438639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a9a12-7a09-43d5-a81b-d4a78b4f52d3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.443453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.444106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.444391 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.444668 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] PATH_INFO: `/v3/projects/55d342946f714da99e6649aa6f2e8801/users/bef0e72461dd4d19bf250fc6542f7c04/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.459699 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d342946f714da99e6649aa6f2e8801, user_id=bef0e72461dd4d19bf250fc6542f7c04, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.463283 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.526912 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1308/2619] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:17 2022] PUT /identity/v3/projects/55d342946f714da99e6649aa6f2e8801/users/bef0e72461dd4d19bf250fc6542f7c04/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:17.537821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.538109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.538414 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.567319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] MFA Rules not processed for user `bef0e72461dd4d19bf250fc6542f7c04`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:17.568695 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:17.632869 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1312/2620] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:17.638084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.700494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe0c2a-01cd-4504-92b5-712148e62914 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.705725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.709975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.709975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.709975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.709975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.709975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.825387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1309/2621] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:17 2022] POST /identity/v3/projects => generated 381 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:17.829540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.843639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefc795-5648-426d-b3d1-5badeef11010 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.853990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:17.860533 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:17.901295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.901916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.902202 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.902510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.903154 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:17.905085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:17.939378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bbb76-26f0-44c6-91de-d2affc6c2eea admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:17.943242 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:17.943843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:17.944121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:17.944392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:17.946170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:17.967345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:18.009848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.011580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.017916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1313/2622] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:17 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:18.021807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:18.025120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1310/2623] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:18.033019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3b92d-9cd5-45a0-bc15-74b142dc93f1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:18.037877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:18.038519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.038922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.039191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.044858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.044858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.052285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1314/2624] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:18.056654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:18.067099 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158496c-a7b0-4a88-852c-3b569fa53310 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:18.071868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:18.072529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.072849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.073135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] PATH_INFO: `/v3/projects/d34b1bafedeb41ff807dce41ccfe9a09/users/bcbc35bb39714537a7485e1aa7a0f499/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.089092 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34b1bafedeb41ff807dce41ccfe9a09, user_id=bcbc35bb39714537a7485e1aa7a0f499, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.091950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.172915 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1311/2625] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:18 2022] PUT /identity/v3/projects/d34b1bafedeb41ff807dce41ccfe9a09/users/bcbc35bb39714537a7485e1aa7a0f499/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:18.176974 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:18.187252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca342c0f-1144-40dd-acbd-4cc1f4056312 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:18.249616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306782a-a04f-4153-8405-128ebd9f4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:18.250923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306782a-a04f-4153-8405-128ebd9f4936 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.251207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306782a-a04f-4153-8405-128ebd9f4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.251476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306782a-a04f-4153-8405-128ebd9f4936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.252122 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306782a-a04f-4153-8405-128ebd9f4936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.253714 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306782a-a04f-4153-8405-128ebd9f4936 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.259484 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:18.267863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1315/2626] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:18.269145 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:18.280739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9033-bcdf-4799-980d-728469a6f87e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:18.285911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:18.286612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.286871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.287142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] PATH_INFO: `/v3/projects/d34b1bafedeb41ff807dce41ccfe9a09/users/bcbc35bb39714537a7485e1aa7a0f499/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.289039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34b1bafedeb41ff807dce41ccfe9a09, user_id=bcbc35bb39714537a7485e1aa7a0f499, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.293027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.303462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1312/2627] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:18 2022] PUT /identity/v3/projects/d34b1bafedeb41ff807dce41ccfe9a09/users/bcbc35bb39714537a7485e1aa7a0f499/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:18.319245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.319536 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.319807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.348829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] MFA Rules not processed for user `bcbc35bb39714537a7485e1aa7a0f499`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:18.350035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:18.416824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1316/2628] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:18.428889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:18.485056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591afc1-f012-4fd4-81ce-44d781d9a9ea admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:18.488713 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:18.489315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.489595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.489867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.498941 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.500394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.510189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1313/2629] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:18.705896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:18.715721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae6c76-7b4b-4a48-9022-47b60cd3caad None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:18.719239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e57e4-4dd5-48a3-92fe-a80e9cd60a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:18.719831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e57e4-4dd5-48a3-92fe-a80e9cd60a0d service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:18.720135 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e57e4-4dd5-48a3-92fe-a80e9cd60a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:18.720400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e57e4-4dd5-48a3-92fe-a80e9cd60a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:18.777422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e57e4-4dd5-48a3-92fe-a80e9cd60a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:18.778889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e57e4-4dd5-48a3-92fe-a80e9cd60a0d service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:18.788147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1317/2630] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:22.645286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:22.646722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:22.710725 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419a22b-c97a-4be4-8510-3f7ecb5abf70 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:22.715814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:22.716435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:22.716712 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:22.716982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] PATH_INFO: `/v3/users/b9c1de6281aa4bfbb58d489315ae1938` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:22.718079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c1de6281aa4bfbb58d489315ae1938)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:22.720042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:22.743074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:22.986020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c1de6281aa4bfbb58d489315ae1938'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.057682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c1de6281aa4bfbb58d489315ae1938'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.058321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c1de6281aa4bfbb58d489315ae1938'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.072197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c1de6281aa4bfbb58d489315ae1938'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.079545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c1de6281aa4bfbb58d489315ae1938'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.085996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1314/2631] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:22 2022] DELETE /identity/v3/users/b9c1de6281aa4bfbb58d489315ae1938 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:23.091288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:23.091762 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:23.091890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:23.111628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:23.112855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:23.172596 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1318/2632] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:23.187143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:23.188604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:23.245489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276c7d9-5ad7-4f45-86f5-673cbd1f9fb6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:23.249020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:23.249631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:23.249916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:23.250192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:23.259192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:23.260654 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:23.269951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1315/2633] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:23.653663 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:23.658772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:23.669067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d9fec-c545-428c-abcd-9aeeeea8cfbc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:23.673920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:23.674630 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:23.674912 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:23.675185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] PATH_INFO: `/v3/projects/09010bacbc1c46228168164dd1cda48a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:23.676322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09010bacbc1c46228168164dd1cda48a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:23.678257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:23.700443 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:23.788541 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09010bacbc1c46228168164dd1cda48a', 'request_id': 'req-f5df2496-3fef-4825-8329-af0322673eb1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.855898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09010bacbc1c46228168164dd1cda48a', 'request_id': 'req-f5df2496-3fef-4825-8329-af0322673eb1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.862085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09010bacbc1c46228168164dd1cda48a', 'request_id': 'req-f5df2496-3fef-4825-8329-af0322673eb1'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.875160 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] The token cache is being invalidate because project 09010bacbc1c46228168164dd1cda48a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:23.875454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.876107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:23.877968 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1319/2634] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:23 2022] DELETE /identity/v3/projects/09010bacbc1c46228168164dd1cda48a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:23.881972 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:23.894585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8767-f2cb-4635-913e-f0004bcd7dd3 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:23.950500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:23.951128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:23.951407 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:23.951679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] PATH_INFO: `/v3/users/289da527111d4166813b3aece74c7699` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:23.952817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=289da527111d4166813b3aece74c7699)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:23.954793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:24.169396 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:24.171641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289da527111d4166813b3aece74c7699'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.234321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289da527111d4166813b3aece74c7699'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.234975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289da527111d4166813b3aece74c7699'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.248157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289da527111d4166813b3aece74c7699'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.254975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289da527111d4166813b3aece74c7699'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.261910 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1316/2635] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:23 2022] DELETE /identity/v3/users/289da527111d4166813b3aece74c7699 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:24.682291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:24.685136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:24.697728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2496-3fef-4825-8329-af0322673eb1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:24.760578 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:24.761280 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:24.761522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:24.761792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] PATH_INFO: `/v3/projects/29ec827eb3c24625a21801da49f1523a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:24.762943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ec827eb3c24625a21801da49f1523a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:24.764870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:24.890962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ec827eb3c24625a21801da49f1523a', 'request_id': 'req-2c49150d-8015-4401-8f62-8f1b8d6c4642'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.958931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ec827eb3c24625a21801da49f1523a', 'request_id': 'req-2c49150d-8015-4401-8f62-8f1b8d6c4642'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.965238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ec827eb3c24625a21801da49f1523a', 'request_id': 'req-2c49150d-8015-4401-8f62-8f1b8d6c4642'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.977659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] The token cache is being invalidate because project 29ec827eb3c24625a21801da49f1523a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:24.978012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.980180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:24.980447 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1320/2636] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:24 2022] DELETE /identity/v3/projects/29ec827eb3c24625a21801da49f1523a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:24.984585 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:24.986350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:24.999895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5898d6-00c1-409a-a6ab-20b219b51383 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:25.055526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:25.056155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:25.056435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:25.056709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] PATH_INFO: `/v3/users/9b48d0d60ba84bd8bc3e7a5b82ab7ff1` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:25.057815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b48d0d60ba84bd8bc3e7a5b82ab7ff1)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:25.059759 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:25.236098 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b48d0d60ba84bd8bc3e7a5b82ab7ff1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:25.324197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b48d0d60ba84bd8bc3e7a5b82ab7ff1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:25.324815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b48d0d60ba84bd8bc3e7a5b82ab7ff1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:25.337313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b48d0d60ba84bd8bc3e7a5b82ab7ff1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:25.344379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b48d0d60ba84bd8bc3e7a5b82ab7ff1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:25.350473 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1317/2637] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:24 2022] DELETE /identity/v3/users/9b48d0d60ba84bd8bc3e7a5b82ab7ff1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:25.783624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:25.785469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:25.800028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49150d-8015-4401-8f62-8f1b8d6c4642 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:25.844565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:25.869857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:25.870545 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:25.870829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:25.871100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] PATH_INFO: `/v3/projects/81c5054d6c264244aa6f077707eb580c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:25.872319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c5054d6c264244aa6f077707eb580c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:25.874272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:26.016811 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c5054d6c264244aa6f077707eb580c', 'request_id': 'req-b06b12db-9223-4356-9d6e-82ff36c68827'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:26.103699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c5054d6c264244aa6f077707eb580c', 'request_id': 'req-b06b12db-9223-4356-9d6e-82ff36c68827'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:26.109881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c5054d6c264244aa6f077707eb580c', 'request_id': 'req-b06b12db-9223-4356-9d6e-82ff36c68827'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:26.121998 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] The token cache is being invalidate because project 81c5054d6c264244aa6f077707eb580c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:26.122294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:26.123112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b06b12db-9223-4356-9d6e-82ff36c68827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:26.124882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1321/2638] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:25 2022] DELETE /identity/v3/projects/81c5054d6c264244aa6f077707eb580c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:30.404179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:30.405628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:30.419034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57422647-5d44-4380-92c1-fc6b260693c8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:30.459137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:30.475708 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:30.476337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:30.476762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:30.477046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] PATH_INFO: `/v3/users/bef0e72461dd4d19bf250fc6542f7c04` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:30.478201 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef0e72461dd4d19bf250fc6542f7c04)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:30.480211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:30.660774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef0e72461dd4d19bf250fc6542f7c04'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:30.737042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef0e72461dd4d19bf250fc6542f7c04'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:30.737665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef0e72461dd4d19bf250fc6542f7c04'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:30.751641 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef0e72461dd4d19bf250fc6542f7c04'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:30.759147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef0e72461dd4d19bf250fc6542f7c04'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:30.765333 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1318/2639] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:30 2022] DELETE /identity/v3/users/bef0e72461dd4d19bf250fc6542f7c04 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:30.770028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:30.770508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:30.770789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:30.791726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:30.792944 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:30.853549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1322/2640] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:30.863251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:30.868125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:30.930107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318d2e0-099f-4617-bed2-a055bd44f942 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:30.935446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:30.936359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:30.936783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:30.937203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:30.950616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:30.952918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:30.965583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1319/2641] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:31.409910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:31.415187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:31.427344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c58231-2857-4771-8660-74c4e6da315e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:31.436205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:31.437156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:31.437339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:31.437571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] PATH_INFO: `/v3/projects/55d342946f714da99e6649aa6f2e8801` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:31.437935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:31.438745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d342946f714da99e6649aa6f2e8801)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:31.440721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:31.447694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6905327-12a1-484e-b72f-7b4af2bdfbea service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:31.451545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:31.452151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:31.452440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:31.452731 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:31.454579 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:31.466901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:31.468381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:31.478476 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1320/2642] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:31.536274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d342946f714da99e6649aa6f2e8801', 'request_id': 'req-2226f466-29f9-4f16-ae9c-a6374def81de'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.604324 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d342946f714da99e6649aa6f2e8801', 'request_id': 'req-2226f466-29f9-4f16-ae9c-a6374def81de'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.609581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:31.610790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d342946f714da99e6649aa6f2e8801', 'request_id': 'req-2226f466-29f9-4f16-ae9c-a6374def81de'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.623020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] The token cache is being invalidate because project 55d342946f714da99e6649aa6f2e8801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:31.623317 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.623964 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.625794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1323/2643] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:31 2022] DELETE /identity/v3/projects/55d342946f714da99e6649aa6f2e8801 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:31.629901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:31.640802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba4260-e6ad-47dc-8556-b63e827206f4 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:31.704730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:31.714889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:31.715956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:31.715956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:31.716197 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] PATH_INFO: `/v3/users/bcbc35bb39714537a7485e1aa7a0f499` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:31.717268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbc35bb39714537a7485e1aa7a0f499)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:31.719261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:31.883692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbc35bb39714537a7485e1aa7a0f499'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.941463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbc35bb39714537a7485e1aa7a0f499'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.942112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbc35bb39714537a7485e1aa7a0f499'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.955898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbc35bb39714537a7485e1aa7a0f499'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.965680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f93b757b-f142-4842-8090-7e9a7f3827f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbc35bb39714537a7485e1aa7a0f499'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:31.972274 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1321/2644] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:31 2022] DELETE /identity/v3/users/bcbc35bb39714537a7485e1aa7a0f499 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:32.765565 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:32.767624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:32.784855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226f466-29f9-4f16-ae9c-a6374def81de admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:32.860516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:32.861157 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:32.861436 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:32.861726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] PATH_INFO: `/v3/projects/d34b1bafedeb41ff807dce41ccfe9a09` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:32.862842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34b1bafedeb41ff807dce41ccfe9a09)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:32.864751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:32.970572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34b1bafedeb41ff807dce41ccfe9a09', 'request_id': 'req-ce286fd7-00b8-4132-ab81-43e14c9d1a38'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:33.028498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34b1bafedeb41ff807dce41ccfe9a09', 'request_id': 'req-ce286fd7-00b8-4132-ab81-43e14c9d1a38'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:33.035210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34b1bafedeb41ff807dce41ccfe9a09', 'request_id': 'req-ce286fd7-00b8-4132-ab81-43e14c9d1a38'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:33.046797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] The token cache is being invalidate because project d34b1bafedeb41ff807dce41ccfe9a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:33.047086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:33.047729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce286fd7-00b8-4132-ab81-43e14c9d1a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:33.049553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1324/2645] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:32 2022] DELETE /identity/v3/projects/d34b1bafedeb41ff807dce41ccfe9a09 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:33.054184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.055255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.055411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.076775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:33.077920 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:33.138147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1322/2646] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:33.146136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.146486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.146760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.167283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:33.168530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:33.176376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1325/2647] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:33.181410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.191405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b651ac-9b69-4e39-ab57-c2679ab55d67 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.196185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.196875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.197226 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.197569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.198659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:33.198659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.200772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:33.212671 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1323/2648] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:33.216778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.226029 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700a06a-6d70-4275-99b8-70796c3df4b2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.230893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.231487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.231764 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.232030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.232673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.234585 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:33.325748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1326/2649] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:33 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:33.330220 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.341625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093f9c-ec69-413c-8cb3-d5ff0bcfcce0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.402121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:33.408984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.409943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.410432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.410762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.411500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.413666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:33.519345 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1324/2650] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:33 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:33.524014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.534891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69355f70-8ce7-4724-93ef-d58a585ce4eb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.544879 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.545676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.546373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.546373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.547163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.548989 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:33.562186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1327/2651] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:33.566272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.580778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f3ac1-9ca8-402d-a963-e746720101c5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.587464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c73b3-e3b1-48d5-967a-cad87043843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.588198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c73b3-e3b1-48d5-967a-cad87043843e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.589594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c73b3-e3b1-48d5-967a-cad87043843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.589594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c73b3-e3b1-48d5-967a-cad87043843e admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.589594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c73b3-e3b1-48d5-967a-cad87043843e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.591614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c73b3-e3b1-48d5-967a-cad87043843e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:33.605124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1325/2652] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:33.609411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.620321 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3659d-aa94-4ac3-9e0d-f87665a5e882 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.625734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.626648 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.627285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.627285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] PATH_INFO: `/v3/projects/33e6a990163440edb5fdaa2d169ae3ab/users/293487448fc741eba487cc73fc24c99f/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.652836 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e6a990163440edb5fdaa2d169ae3ab, user_id=293487448fc741eba487cc73fc24c99f, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.656170 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:33.749704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1328/2653] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:33 2022] PUT /identity/v3/projects/33e6a990163440edb5fdaa2d169ae3ab/users/293487448fc741eba487cc73fc24c99f/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:33.752305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.753067 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.753638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.794118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] MFA Rules not processed for user `293487448fc741eba487cc73fc24c99f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:33.794118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:33.866679 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1326/2654] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:33.871403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:33.891087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:33.936815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280ada-7330-4f6e-98bb-818259449fd1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:33.942733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:33.943432 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:33.943718 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:33.943988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:33.944625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:33.947111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:34.036384 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1329/2655] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:33 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:34.041033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:34.052454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef6da36-8309-423b-b9e7-e2a27d45b901 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:34.112877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:34.114143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.114768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.115052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.115726 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:34.118998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:34.229644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1327/2656] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:34 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:34.233604 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:34.243649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54c67b-f9a7-4a0e-9b56-a35c7e57ff64 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:34.251299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:34.251299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.251299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.251299 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.251863 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:34.253161 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:34.393278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1330/2657] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:34.395416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:34.406100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25058dcf-fdac-4711-8b16-54557144e263 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:34.410995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:34.411655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.411939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.412212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] PATH_INFO: `/v3/projects/5cc7b09e60d54e8e913880526365afe5/users/a9241404dcee437cb501e83208309b63/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.428431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc7b09e60d54e8e913880526365afe5, user_id=a9241404dcee437cb501e83208309b63, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:34.431783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:34.497520 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1328/2658] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:34 2022] PUT /identity/v3/projects/5cc7b09e60d54e8e913880526365afe5/users/a9241404dcee437cb501e83208309b63/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:34.501519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:34.511548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee863550-9890-4fed-8d01-3e218d4bd1e2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:34.584147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad0c09-f94f-44e3-881a-75476613d607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:34.584930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad0c09-f94f-44e3-881a-75476613d607 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.585286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad0c09-f94f-44e3-881a-75476613d607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.586209 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad0c09-f94f-44e3-881a-75476613d607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.587174 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad0c09-f94f-44e3-881a-75476613d607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:34.594616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad0c09-f94f-44e3-881a-75476613d607 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:34.601130 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1331/2659] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:34.605717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:34.616986 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af469d6b-4a36-4292-a236-a8775c7f2617 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:34.621762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:34.623240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.623240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.623240 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] PATH_INFO: `/v3/projects/5cc7b09e60d54e8e913880526365afe5/users/a9241404dcee437cb501e83208309b63/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.624928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc7b09e60d54e8e913880526365afe5, user_id=a9241404dcee437cb501e83208309b63, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:34.627655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:34.641545 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1329/2660] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:34 2022] PUT /identity/v3/projects/5cc7b09e60d54e8e913880526365afe5/users/a9241404dcee437cb501e83208309b63/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:34.652319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.652617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.652889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.701333 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] MFA Rules not processed for user `a9241404dcee437cb501e83208309b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:34.702621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:34.862868 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1332/2661] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:34.873933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:34.875720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:34.912704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:34.970252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171313c0-a958-47d9-a20a-f37efe54f6eb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:34.974995 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:34.975810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:34.976846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:34.977495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:34.989502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:34.991312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:35.008504 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1330/2662] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:35.134923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:35.149330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fd82e-39bf-4932-8efd-b04b04be94bd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:35.153214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:35.153871 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:35.154191 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:35.165188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:35.178633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:35.229502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:35.231032 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:35.240556 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1333/2663] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:35.431976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:35.433493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:35.502888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890455f-754f-47e8-b99f-0df6d1c1425f service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:35.507452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:35.508124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:35.508402 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:35.508677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] PATH_INFO: `/v3/users/29b4f0836bbf4684bdc6177e42b88b20` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:35.509804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b4f0836bbf4684bdc6177e42b88b20)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:35.511770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:35.726669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b4f0836bbf4684bdc6177e42b88b20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:35.782503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b4f0836bbf4684bdc6177e42b88b20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:35.784474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b4f0836bbf4684bdc6177e42b88b20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:35.796977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b4f0836bbf4684bdc6177e42b88b20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:35.804081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5bfda70f-a380-43e7-abab-db528922a49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b4f0836bbf4684bdc6177e42b88b20'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:35.810843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1331/2664] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:35 2022] DELETE /identity/v3/users/29b4f0836bbf4684bdc6177e42b88b20 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:36.087835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.104220 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59967-bf45-4d43-94a3-79d322d05a04 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.161462 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.162133 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.162379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.162652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] PATH_INFO: `/v3/projects/e09bed4b9f9741a8ab110a2676baee7f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.163876 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e09bed4b9f9741a8ab110a2676baee7f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.165755 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.280784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e09bed4b9f9741a8ab110a2676baee7f', 'request_id': 'req-d18127eb-e340-4438-96e4-bb0aba716c53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:36.342400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e09bed4b9f9741a8ab110a2676baee7f', 'request_id': 'req-d18127eb-e340-4438-96e4-bb0aba716c53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:36.348498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e09bed4b9f9741a8ab110a2676baee7f', 'request_id': 'req-d18127eb-e340-4438-96e4-bb0aba716c53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:36.359992 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] The token cache is being invalidate because project e09bed4b9f9741a8ab110a2676baee7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:36.360283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:36.360916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d18127eb-e340-4438-96e4-bb0aba716c53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:36.362437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1334/2665] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:36 2022] DELETE /identity/v3/projects/e09bed4b9f9741a8ab110a2676baee7f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:36.367970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.368259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.368541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.389649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:36.390892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:36.423933 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:36.450438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1332/2666] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:36.458504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.458796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.459073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.479587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:36.481109 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:36.489310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1335/2667] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:36.494378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.504472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f5567-3f4c-4ec5-b278-d8f899ad81d5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.509339 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.510040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.510450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.510828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.511580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:36.511984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.513799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.525245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1333/2668] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 03 01:41:36.529421 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.540004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a37ce-24a7-4096-9d9a-ddaea07f8175 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.544840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.545431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.545712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.549006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.549006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.549006 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.624376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1336/2669] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:36 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:36.628511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.638856 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379f446-9a0a-4969-923c-7c7f93d90643 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.697474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.703285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.703285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.703285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.703285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.703285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.771327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1334/2670] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:36 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:36.776054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.786675 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf527e-b75b-4e31-85f1-b57798a1c55c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.792051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.792600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.792882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.793159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.793895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.795521 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.805567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1337/2671] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:36.809613 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.820176 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6da9b-bbbf-40c2-a73c-eab325384f7b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.824897 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10754e-60ef-46f9-b291-609b53a605cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.825500 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10754e-60ef-46f9-b291-609b53a605cf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.825786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10754e-60ef-46f9-b291-609b53a605cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.826056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10754e-60ef-46f9-b291-609b53a605cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.826859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10754e-60ef-46f9-b291-609b53a605cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.828457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10754e-60ef-46f9-b291-609b53a605cf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.840321 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1335/2672] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:36.844272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:36.849445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:36.855348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e660619-c178-4496-9c9c-c30f15e48333 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:36.860019 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:36.860664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.860941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.861207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] PATH_INFO: `/v3/projects/154b559256be4252aa82eb7c8c636f7e/users/9f3ab78a28a64eb6a7990284974eb399/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.875937 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154b559256be4252aa82eb7c8c636f7e, user_id=9f3ab78a28a64eb6a7990284974eb399, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:36.878796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:36.931984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1338/2673] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:36 2022] PUT /identity/v3/projects/154b559256be4252aa82eb7c8c636f7e/users/9f3ab78a28a64eb6a7990284974eb399/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:36.944069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:36.944379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:36.944660 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:36.973384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] MFA Rules not processed for user `9f3ab78a28a64eb6a7990284974eb399`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:36.974629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:37.042365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1336/2674] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:37.054238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:37.056729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:37.134150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa30a4f-710c-4f44-8d5c-142686b8da62 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:37.139794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304f93b-adf9-4aa7-a6e4-b0b924db1000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:37.140398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304f93b-adf9-4aa7-a6e4-b0b924db1000 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:37.140679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304f93b-adf9-4aa7-a6e4-b0b924db1000 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:37.140949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304f93b-adf9-4aa7-a6e4-b0b924db1000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:37.148896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304f93b-adf9-4aa7-a6e4-b0b924db1000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:37.150518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304f93b-adf9-4aa7-a6e4-b0b924db1000 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:37.159237 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1339/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:37.387784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:37.427076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:37.450744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc33cbd-26aa-415e-9d1f-020e08a08554 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:37.455511 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:37.456318 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:37.456650 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:37.457253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] PATH_INFO: `/v3/users/9f3ab78a28a64eb6a7990284974eb399` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:37.458526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3ab78a28a64eb6a7990284974eb399)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:37.460461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:37.647673 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3ab78a28a64eb6a7990284974eb399'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:37.708268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3ab78a28a64eb6a7990284974eb399'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:37.708903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3ab78a28a64eb6a7990284974eb399'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:37.721543 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3ab78a28a64eb6a7990284974eb399'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:37.728666 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3ab78a28a64eb6a7990284974eb399'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:37.734634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1337/2676] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:37 2022] DELETE /identity/v3/users/9f3ab78a28a64eb6a7990284974eb399 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:37.739075 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:37.746288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:37.746288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:37.759626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:37.760833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:37.821744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1340/2677] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:37.831522 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:37.832926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:37.891326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088830db-6882-467f-9303-178026ef1a19 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:37.894956 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:37.895631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:37.895911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:37.896179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:37.905581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:37.906999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:37.917630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1338/2678] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:38.292380 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:38.293931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.311485 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80453f-952a-4953-8b35-1393bfaf1553 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:38.317065 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:38.317748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.317942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.318306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] PATH_INFO: `/v3/projects/154b559256be4252aa82eb7c8c636f7e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.319355 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154b559256be4252aa82eb7c8c636f7e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:38.321628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:38.353499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:38.355036 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.367711 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65875aa4-a8be-40f6-84c2-640131855707 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:38.372183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:38.372795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.373077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.373347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] PATH_INFO: `/v3/users/293487448fc741eba487cc73fc24c99f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.374535 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=293487448fc741eba487cc73fc24c99f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:38.376445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:38.429759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154b559256be4252aa82eb7c8c636f7e', 'request_id': 'req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.485364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154b559256be4252aa82eb7c8c636f7e', 'request_id': 'req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.487652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:38.491799 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154b559256be4252aa82eb7c8c636f7e', 'request_id': 'req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.502852 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] The token cache is being invalidate because project 154b559256be4252aa82eb7c8c636f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:38.503138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.503756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d96b4f02-fcfb-43c2-bfb7-80a6a828c1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.505205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1341/2679] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:38 2022] DELETE /identity/v3/projects/154b559256be4252aa82eb7c8c636f7e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:38.509904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe50e0-be03-44ca-8653-2da2b68d053e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.510192 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe50e0-be03-44ca-8653-2da2b68d053e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.510506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe50e0-be03-44ca-8653-2da2b68d053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.529982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2ebe50e0-be03-44ca-8653-2da2b68d053e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:38.531201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2ebe50e0-be03-44ca-8653-2da2b68d053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.551386 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '293487448fc741eba487cc73fc24c99f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.555222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:38.591088 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1342/2680] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:38.599066 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.599356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.599632 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.611630 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '293487448fc741eba487cc73fc24c99f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.612361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '293487448fc741eba487cc73fc24c99f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.618560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:38.619768 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.625590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '293487448fc741eba487cc73fc24c99f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.627864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1343/2681] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:38.632048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:38.633110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9fe6cd81-6599-4528-9eb7-da7a151d46b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '293487448fc741eba487cc73fc24c99f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:38.642716 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1339/2682] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:38 2022] DELETE /identity/v3/users/293487448fc741eba487cc73fc24c99f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:38.643508 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f88b8-924c-4b47-806a-ae7451159dd3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:38.645509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.645792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.646076 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.665225 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:38.666458 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.700744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:38.701464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.701839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.702410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.703156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:38.703560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:38.705215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:38.721128 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1344/2683] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:38.722073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:38.723677 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.726888 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1340/2684] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:38.736214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819b754-158d-4981-9449-f45bebd8145f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:38.736371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:38.737609 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:38.740810 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:38.741398 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.741673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.741973 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.742634 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:38.744537 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:38.796411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f23e67-8d51-45fc-8a76-29658232c551 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:38.800148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:38.800754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.801029 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.801299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.811065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:38.812612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:38.822815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1341/2685] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:38.855997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1345/2686] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:38 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:38.857459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:38.867243 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c52886-cc83-4903-a0cb-296d9889c695 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:38.926203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:38.926842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:38.927124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:38.927399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:38.928027 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:38.930576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.018002 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1342/2687] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:38 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:39.022050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.032491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7392763-e426-42d1-a460-d667ec904afb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.044475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.044475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.044704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.045003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.045654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.047442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.059385 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1346/2688] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:39.063767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.068983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:39.075327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984187eb-3eff-48ea-b18a-34db63c062a8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.080259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.080864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.081248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.081517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.082155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.083814 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.095062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1343/2689] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:39.099139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.109424 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b3947-88e7-47b6-9a14-385a0debeb95 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.122639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b464e0d-11dc-4c33-8d31-0d7ea9d09bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.124434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b464e0d-11dc-4c33-8d31-0d7ea9d09bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.124434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b464e0d-11dc-4c33-8d31-0d7ea9d09bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.124434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b464e0d-11dc-4c33-8d31-0d7ea9d09bbc admin admin] PATH_INFO: `/v3/projects/9e65a294c74f40e390c1f7832f7e8a64/users/5be3656e855843d286f552b3d3a422da/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.138891 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b464e0d-11dc-4c33-8d31-0d7ea9d09bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e65a294c74f40e390c1f7832f7e8a64, user_id=5be3656e855843d286f552b3d3a422da, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.141679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b464e0d-11dc-4c33-8d31-0d7ea9d09bbc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.224289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1347/2690] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:39 2022] PUT /identity/v3/projects/9e65a294c74f40e390c1f7832f7e8a64/users/5be3656e855843d286f552b3d3a422da/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:39.224553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.226996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:39.236662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.236951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.237228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.240390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aedbb24-7c3c-46e5-b4f8-0be37bd4232a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.274743 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] MFA Rules not processed for user `5be3656e855843d286f552b3d3a422da`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:39.275995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:39.282962 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:39.306010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.306686 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.307006 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.307246 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] PATH_INFO: `/v3/projects/33e6a990163440edb5fdaa2d169ae3ab` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.308383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e6a990163440edb5fdaa2d169ae3ab)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.310376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.347680 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1348/2691] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:39.352676 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.364036 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1397004-2b50-4d22-bca2-ef5c869b5af6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.369201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.369796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.370080 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.370358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.371023 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.372951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.514660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1349/2692] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:39 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:39.515691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.521743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e6a990163440edb5fdaa2d169ae3ab', 'request_id': 'req-7434c920-6df7-4202-ab14-32051aa9c98b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.578514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd552eb-a50e-4635-8f9e-64a12ec973d9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.583195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.583797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.584073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.584340 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.584965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.587606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.605384 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e6a990163440edb5fdaa2d169ae3ab', 'request_id': 'req-7434c920-6df7-4202-ab14-32051aa9c98b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.611551 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e6a990163440edb5fdaa2d169ae3ab', 'request_id': 'req-7434c920-6df7-4202-ab14-32051aa9c98b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.623205 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] The token cache is being invalidate because project 33e6a990163440edb5fdaa2d169ae3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:39.623493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.624133 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.625621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1344/2693] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:39 2022] DELETE /identity/v3/projects/33e6a990163440edb5fdaa2d169ae3ab => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:39.629405 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.635265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:39.683931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1350/2694] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:39 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:39.684954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.686692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:39.699064 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434c920-6df7-4202-ab14-32051aa9c98b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.700021 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a41f3a-82e1-4481-9a76-9d9eddfedd22 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.703784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.704418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.704665 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.704804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.704957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] PATH_INFO: `/v3/users/a9241404dcee437cb501e83208309b63` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.705261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.705558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.705820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.706084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9241404dcee437cb501e83208309b63)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.706567 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.708056 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.708187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.719267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1351/2695] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:39.723203 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.732011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:39.732888 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87917262-633b-4483-9f50-0aaf07608334 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.737476 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.742231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.742231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.742231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.742231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.742231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.751726 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1352/2696] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:39.755707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.765970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236a622-86da-4dc4-9231-9583fa2cccab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:39.770502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd591d-227a-4538-8560-608eb354f6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:39.771120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd591d-227a-4538-8560-608eb354f6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.771400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd591d-227a-4538-8560-608eb354f6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.771673 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd591d-227a-4538-8560-608eb354f6f6 admin admin] PATH_INFO: `/v3/projects/89bb1810d7eb4a63a9c69dd98268442b/users/8107587901bd4d34bc0b95621b9248a7/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.786197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd591d-227a-4538-8560-608eb354f6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89bb1810d7eb4a63a9c69dd98268442b, user_id=8107587901bd4d34bc0b95621b9248a7, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:39.789035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd591d-227a-4538-8560-608eb354f6f6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:39.853012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1353/2697] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:39 2022] PUT /identity/v3/projects/89bb1810d7eb4a63a9c69dd98268442b/users/8107587901bd4d34bc0b95621b9248a7/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:39.863275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:39.863563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:39.863835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:39.865272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9241404dcee437cb501e83208309b63'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.891470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] MFA Rules not processed for user `8107587901bd4d34bc0b95621b9248a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:39.892689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:39.909880 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:39.930387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9241404dcee437cb501e83208309b63'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.931124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9241404dcee437cb501e83208309b63'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.943927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9241404dcee437cb501e83208309b63'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.951901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9241404dcee437cb501e83208309b63'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:39.958028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1345/2698] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:39 2022] DELETE /identity/v3/users/a9241404dcee437cb501e83208309b63 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:39.961120 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1354/2699] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:39.962733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:39.964107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.029720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ba0cf-6cbf-4e60-a272-82b44c7154cd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.034934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.035553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.035838 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.036113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.036768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.038949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.155894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1346/2700] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:39 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:40.160311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.231431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793a0cc-b7f9-4e7f-b7f6-7e7385251d65 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.293668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.294515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.294904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.295442 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.296184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.298210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.422821 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1355/2701] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:40 2022] POST /identity/v3/users => generated 299 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:40.426929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.437601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bc6d4-11e6-4ec4-92df-9b15abb2a7cb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.446658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.446658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.446658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.446658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.447055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.447055 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.448499 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.450304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.456816 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1347/2702] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:40.461442 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.470546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328522e-40a7-4002-9bd2-86fd1457374e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.472981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcb4c0-2bcf-4f6b-9b0e-1c926ebdda13 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.475283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.475916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.476765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.477210 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] PATH_INFO: `/v3/projects/5cc7b09e60d54e8e913880526365afe5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.478518 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc7b09e60d54e8e913880526365afe5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.479401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.480204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.480459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.480670 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.481069 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] PATH_INFO: `/v3/projects/487fe37416e14ad0b66a166022a87a79/users/f09981d845294c47970ba99f74bb784b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.497115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=487fe37416e14ad0b66a166022a87a79, user_id=f09981d845294c47970ba99f74bb784b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.499984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.565783 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1348/2703] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:40 2022] PUT /identity/v3/projects/487fe37416e14ad0b66a166022a87a79/users/f09981d845294c47970ba99f74bb784b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:40.569674 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.581760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc7b09e60d54e8e913880526365afe5', 'request_id': 'req-71b35c6a-8454-410f-aa77-613361d40eda'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:40.583532 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:40.620437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:40.631113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a859-733f-4aa1-a369-8296e7715131 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.634833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc7b09e60d54e8e913880526365afe5', 'request_id': 'req-71b35c6a-8454-410f-aa77-613361d40eda'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:40.641454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc7b09e60d54e8e913880526365afe5', 'request_id': 'req-71b35c6a-8454-410f-aa77-613361d40eda'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:40.653674 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] The token cache is being invalidate because project 5cc7b09e60d54e8e913880526365afe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:40.653965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:40.654623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-71b35c6a-8454-410f-aa77-613361d40eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:40.656131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1356/2704] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:40 2022] DELETE /identity/v3/projects/5cc7b09e60d54e8e913880526365afe5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:40.661001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5a3b3-36b4-41e2-85a0-8f6f1138e3d8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.661289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5a3b3-36b4-41e2-85a0-8f6f1138e3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.661561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5a3b3-36b4-41e2-85a0-8f6f1138e3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.681245 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6da5a3b3-36b4-41e2-85a0-8f6f1138e3d8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:40.682602 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6da5a3b3-36b4-41e2-85a0-8f6f1138e3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.688173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.688787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.689191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.689470 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.690111 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.691735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.702131 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1349/2705] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:40.706127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.707876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.719876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03885e-c8b4-4e29-abb1-0798bf9c645b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.724562 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6ef70-267f-461a-a75a-d20377cfacb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.725204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ef70-267f-461a-a75a-d20377cfacb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.725485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ef70-267f-461a-a75a-d20377cfacb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.725756 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ef70-267f-461a-a75a-d20377cfacb9 admin admin] PATH_INFO: `/v3/projects/487fe37416e14ad0b66a166022a87a79/users/f09981d845294c47970ba99f74bb784b/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.727818 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6ef70-267f-461a-a75a-d20377cfacb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=487fe37416e14ad0b66a166022a87a79, user_id=f09981d845294c47970ba99f74bb784b, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.730531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6ef70-267f-461a-a75a-d20377cfacb9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.741228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1350/2706] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:40 2022] PUT /identity/v3/projects/487fe37416e14ad0b66a166022a87a79/users/f09981d845294c47970ba99f74bb784b/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:40.745509 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1357/2707] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:40.751730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.752019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.752293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.754228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.754551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.754833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.775627 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:40.777159 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.782786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] MFA Rules not processed for user `f09981d845294c47970ba99f74bb784b`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:40.784008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.844598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1358/2708] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:40.848707 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.856574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1351/2709] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:40.859874 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91827d5f-1f28-47d5-8e99-e184d3e4a852 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.864711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.865347 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.865623 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.865892 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.866664 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:40.867555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.868139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.869181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.870284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:40.881113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1359/2710] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:40.885612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:40.896167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5cd45-17c0-4114-ad42-decbaca0023a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.900778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.901363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.901626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.902031 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.902624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.904513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.927963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70c7b8-7125-4264-897d-b281b344ca6d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:40.932392 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:40.932998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:40.933276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:40.933571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:40.942492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:40.943926 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:40.953189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1352/2711] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:40.990472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1360/2712] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:40.996750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.007523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a86bd0-c938-49c2-82c4-3d66450667a7 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.012550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:41.066089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.066724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.067003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.067270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.067934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.069849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.172362 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1353/2713] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:40 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:41.175920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.187017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cec3c-69c1-4302-880e-21cf98201b0b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.191619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.192212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.192560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.192751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.193389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.195054 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.204845 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1361/2714] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:41.208734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.219179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bddf0e-8080-4973-9db0-c9dd5c6db95c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.223859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fc429-dbc8-4d2c-ab18-6e24bf50b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.224459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fc429-dbc8-4d2c-ab18-6e24bf50b42a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.224740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fc429-dbc8-4d2c-ab18-6e24bf50b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.231878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fc429-dbc8-4d2c-ab18-6e24bf50b42a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.231878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fc429-dbc8-4d2c-ab18-6e24bf50b42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.231878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fc429-dbc8-4d2c-ab18-6e24bf50b42a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.237932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1354/2715] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:41.242050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.254699 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c38cfd-e817-4100-a851-f830c1cd433a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.257275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.257927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.258205 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.258527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] PATH_INFO: `/v3/projects/8b69af8620c9413cac0f3ac7f3006789/users/166719b165af4b9facd92ee1f88bd0e6/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.274148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b69af8620c9413cac0f3ac7f3006789, user_id=166719b165af4b9facd92ee1f88bd0e6, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.277713 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.349081 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1362/2716] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:41 2022] PUT /identity/v3/projects/8b69af8620c9413cac0f3ac7f3006789/users/166719b165af4b9facd92ee1f88bd0e6/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:41.360419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.360715 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.360998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.389284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] MFA Rules not processed for user `166719b165af4b9facd92ee1f88bd0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:41.390598 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:41.454493 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1355/2717] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:41.458954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.469895 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:41.522771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112603ef-94ae-4a3e-a6eb-7d9f5e7c59fe admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.528208 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.528807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.529093 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.529364 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.530028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.531951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.612953 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1363/2718] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:41 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:41.617108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.627821 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba5132-a801-4c13-ab07-73d81904b7ca None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.684834 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.685494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.685773 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.686039 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.686695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.688691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.788341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1356/2719] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:41 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:41.796692 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.802832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7f690-b961-4f03-a8f2-7f23de6ea226 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.807659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.808263 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.808542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.808816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.809524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.811145 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.824543 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1364/2720] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:41.830311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.842573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2727f70-0594-4be8-ad2c-c284a5186c6b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.847472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.848161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.848523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.848873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.849601 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.855798 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.857255 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:41.871019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1357/2721] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:41.873794 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.873958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:41.875578 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:41.884095 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80255708-5c14-40cb-935e-661d6d1418a4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.889238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c3130-e62f-4afe-9856-b857564bb736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.889921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c3130-e62f-4afe-9856-b857564bb736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.890198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c3130-e62f-4afe-9856-b857564bb736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.890498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c3130-e62f-4afe-9856-b857564bb736 admin admin] PATH_INFO: `/v3/projects/2f9d67539f04454d83b3a50edb6747af/users/333c3f4c6bea430fb80963ab9f035791/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.905616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c3130-e62f-4afe-9856-b857564bb736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9d67539f04454d83b3a50edb6747af, user_id=333c3f4c6bea430fb80963ab9f035791, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:41.908315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c3130-e62f-4afe-9856-b857564bb736 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:41.933247 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb39865-ec8e-4e1e-8ba3-42f2860bbfe9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:41.936916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:41.937515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.937794 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.938063 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:41.939891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:41.971300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1365/2722] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:41 2022] PUT /identity/v3/projects/2f9d67539f04454d83b3a50edb6747af/users/333c3f4c6bea430fb80963ab9f035791/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:41.982258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:41.982579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:41.982850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.000057 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.001517 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.010606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1358/2723] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:42.011965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] MFA Rules not processed for user `333c3f4c6bea430fb80963ab9f035791`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:42.013246 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:42.077807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1366/2724] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:42.082327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.093348 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675bb5d-72dd-4280-bd20-402fde838d2f service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.151884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.153445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.153445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.153445 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.153747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.155664 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.258538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1359/2725] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:42 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:42.263320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.274090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0d3de-bc4c-46ee-b0a6-db52443c1e27 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.324434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:42.336261 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.336975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.338025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.338389 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.339044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.340971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.440842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1367/2726] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:42 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:42.444890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.455760 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb687ea-0af5-4746-803a-07d9604291f2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.461248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.461248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.461248 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.461771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.462781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.464400 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.475341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1360/2727] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:42.479733 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.489621 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838b39f-f9ed-491c-947b-5a2b475863a7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.494323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.494993 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.495316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.495616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] PATH_INFO: `/v3/projects/36cb8446141e4869b95fc6de18b10d6c/users/23bb928d0b95440597a4374034b17a97/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.510622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cb8446141e4869b95fc6de18b10d6c, user_id=23bb928d0b95440597a4374034b17a97, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.513186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.612186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1368/2728] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:42 2022] PUT /identity/v3/projects/36cb8446141e4869b95fc6de18b10d6c/users/23bb928d0b95440597a4374034b17a97/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:42.618874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.629028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af53a-03d8-4497-b582-02c14cf4c50a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.650803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:42.688166 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c3c27-c695-4980-b5bf-2bb4bab3cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.688977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3c27-c695-4980-b5bf-2bb4bab3cdfe admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.689294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3c27-c695-4980-b5bf-2bb4bab3cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.689573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c3c27-c695-4980-b5bf-2bb4bab3cdfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.690376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3c27-c695-4980-b5bf-2bb4bab3cdfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.692007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c3c27-c695-4980-b5bf-2bb4bab3cdfe admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.705665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1361/2729] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:42.708003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.717506 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4803e-8e0d-4449-b036-b4bc9ba8f09e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.722149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.722839 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.723121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.723391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] PATH_INFO: `/v3/projects/36cb8446141e4869b95fc6de18b10d6c/users/23bb928d0b95440597a4374034b17a97/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.725298 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cb8446141e4869b95fc6de18b10d6c, user_id=23bb928d0b95440597a4374034b17a97, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.727965 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.738002 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1369/2730] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:42 2022] PUT /identity/v3/projects/36cb8446141e4869b95fc6de18b10d6c/users/23bb928d0b95440597a4374034b17a97/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:42.749787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.750081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.751193 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.785701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] MFA Rules not processed for user `23bb928d0b95440597a4374034b17a97`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:42.785701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:42.853890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1362/2731] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:42.865840 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:42.923172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4beb00-733b-459c-9cd6-e412c36526c9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:42.926790 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:42.927382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:42.927662 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:42.927930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:42.937423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:42.938885 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:42.948816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1370/2732] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:43.044300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:43.053932 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b454381-7731-4686-94d8-b7067e8e3765 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:43.058251 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:43.058875 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:43.059151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:43.059419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:43.115563 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:43.117172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:43.126848 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1363/2733] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:46.923307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:46.932614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d999c6-4238-4797-9f7f-d35adc6b4324 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:46.936235 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:46.936820 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:46.937094 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:46.937358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:46.939085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:46.998504 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:47.000815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877581a9-913a-41c7-86d6-af190a8e9211 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:47.014928 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1371/2734] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:47.191735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:47.240424 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:47.256824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942d51a-6da2-4c59-ae03-430a94e3d10e service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:47.261549 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:47.262172 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:47.262491 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:47.262765 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] PATH_INFO: `/v3/users/166719b165af4b9facd92ee1f88bd0e6` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:47.263873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166719b165af4b9facd92ee1f88bd0e6)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:47.265792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:47.483041 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166719b165af4b9facd92ee1f88bd0e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:47.558483 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166719b165af4b9facd92ee1f88bd0e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:47.559141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166719b165af4b9facd92ee1f88bd0e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:47.572156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166719b165af4b9facd92ee1f88bd0e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:47.579148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166719b165af4b9facd92ee1f88bd0e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:47.585004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1364/2735] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:47 2022] DELETE /identity/v3/users/166719b165af4b9facd92ee1f88bd0e6 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:47.590575 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:47.590830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:47.591104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:47.610226 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:47.611460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:47.650282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:47.673213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1372/2736] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:47.683967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:47.685432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:47.746722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7b42-c8aa-40f8-a4e4-ac479da77b83 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:47.750261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:47.750894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:47.751177 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:47.751446 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:47.761054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:47.762524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:47.771887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1365/2737] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:48.202469 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:48.203985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:48.217339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0247f2-aaaa-4556-a9a1-e31a58a5ca43 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:48.222028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:48.222691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:48.222979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:48.223251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] PATH_INFO: `/v3/projects/8b69af8620c9413cac0f3ac7f3006789` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:48.224403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b69af8620c9413cac0f3ac7f3006789)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:48.226326 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:48.424478 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b69af8620c9413cac0f3ac7f3006789', 'request_id': 'req-3e7d5dc1-7513-4d08-aca2-aad971928e8b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.565977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b69af8620c9413cac0f3ac7f3006789', 'request_id': 'req-3e7d5dc1-7513-4d08-aca2-aad971928e8b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.572887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b69af8620c9413cac0f3ac7f3006789', 'request_id': 'req-3e7d5dc1-7513-4d08-aca2-aad971928e8b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.583906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] The token cache is being invalidate because project 8b69af8620c9413cac0f3ac7f3006789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:48.584201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.584834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.586315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1373/2738] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:48 2022] DELETE /identity/v3/projects/8b69af8620c9413cac0f3ac7f3006789 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:48.591065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:48.601954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07663d-70ea-4f48-98eb-46398d39ef96 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:48.662748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:48.666925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:48.666925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:48.666925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] PATH_INFO: `/v3/users/333c3f4c6bea430fb80963ab9f035791` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:48.666925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=333c3f4c6bea430fb80963ab9f035791)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:48.667306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:48.672432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:48.831477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333c3f4c6bea430fb80963ab9f035791'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.873959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333c3f4c6bea430fb80963ab9f035791'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.874626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333c3f4c6bea430fb80963ab9f035791'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.888042 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333c3f4c6bea430fb80963ab9f035791'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.895680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333c3f4c6bea430fb80963ab9f035791'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:48.902135 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1366/2739] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:48 2022] DELETE /identity/v3/users/333c3f4c6bea430fb80963ab9f035791 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:49.264323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:49.265848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:49.281491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5dc1-7513-4d08-aca2-aad971928e8b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:49.345311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:49.345951 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:49.346231 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:49.346551 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] PATH_INFO: `/v3/projects/2f9d67539f04454d83b3a50edb6747af` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:49.347703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9d67539f04454d83b3a50edb6747af)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:49.349671 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:49.451307 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9d67539f04454d83b3a50edb6747af', 'request_id': 'req-78dc933b-b6cb-4156-9029-ffb4ac1b4225'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.514526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9d67539f04454d83b3a50edb6747af', 'request_id': 'req-78dc933b-b6cb-4156-9029-ffb4ac1b4225'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.520519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9d67539f04454d83b3a50edb6747af', 'request_id': 'req-78dc933b-b6cb-4156-9029-ffb4ac1b4225'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.522500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:49.532539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] The token cache is being invalidate because project 2f9d67539f04454d83b3a50edb6747af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:49.532829 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.533455 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.535233 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1374/2740] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:49 2022] DELETE /identity/v3/projects/2f9d67539f04454d83b3a50edb6747af => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:49.539224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:49.541472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:49.554034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b076-37e8-451b-a6a4-93611238d06c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:49.610372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:49.611046 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:49.611278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:49.611606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] PATH_INFO: `/v3/users/23bb928d0b95440597a4374034b17a97` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:49.612732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bb928d0b95440597a4374034b17a97)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:49.614745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:49.765734 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bb928d0b95440597a4374034b17a97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.826594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bb928d0b95440597a4374034b17a97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.827353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bb928d0b95440597a4374034b17a97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.841434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bb928d0b95440597a4374034b17a97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.848319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-792fa4bf-a8f1-4b57-9644-d311143d5830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bb928d0b95440597a4374034b17a97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:49.854461 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1367/2741] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:49 2022] DELETE /identity/v3/users/23bb928d0b95440597a4374034b17a97 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:50.251466 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:50.252954 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:50.268519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc933b-b6cb-4156-9029-ffb4ac1b4225 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:50.331420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:50.332074 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:50.332354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:50.332626 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] PATH_INFO: `/v3/projects/36cb8446141e4869b95fc6de18b10d6c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:50.333833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cb8446141e4869b95fc6de18b10d6c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:50.335780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:50.509825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cb8446141e4869b95fc6de18b10d6c', 'request_id': 'req-936f1451-5da6-4960-abb7-55962aa3b9a7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:50.611864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cb8446141e4869b95fc6de18b10d6c', 'request_id': 'req-936f1451-5da6-4960-abb7-55962aa3b9a7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:50.621275 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cb8446141e4869b95fc6de18b10d6c', 'request_id': 'req-936f1451-5da6-4960-abb7-55962aa3b9a7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:50.634238 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] The token cache is being invalidate because project 36cb8446141e4869b95fc6de18b10d6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:50.634563 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:50.635318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-936f1451-5da6-4960-abb7-55962aa3b9a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:50.636950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1375/2742] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:50 2022] DELETE /identity/v3/projects/36cb8446141e4869b95fc6de18b10d6c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:50.643740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:50.644222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:50.644796 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:50.658026 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:50.670070 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:50.671317 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:50.729931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1368/2743] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:50.738362 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:50.738655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:50.738931 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:50.759453 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:50.760672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:50.768655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1376/2744] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:50.773157 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:50.782729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6787a-37bd-4811-a2f5-094c5bd2079e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:50.787580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:50.788179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:50.788459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:50.788733 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:50.789379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:50.789682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:50.791829 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:50.803515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1369/2745] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:50.808898 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:50.818370 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b7d74-ab33-4022-91f7-64c879c9ae55 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:50.824417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:50.825062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:50.825342 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:50.825614 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:50.826253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:50.828278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:50.962490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1377/2746] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:50 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:50.968892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:50.979896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33321390-daab-47d0-8b5e-62121025775c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.017393 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.019188 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:51.042651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.043258 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.043536 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.043805 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.044436 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.046372 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.080916 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:51.084683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54d818-990a-4b11-bae3-502eaf81abc4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.090030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.090671 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.090949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.091215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] PATH_INFO: `/v3/users/5be3656e855843d286f552b3d3a422da` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.092314 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be3656e855843d286f552b3d3a422da)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.094241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.124091 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1370/2747] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:50 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:51.128165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.137736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ece415-63b1-42a5-b411-bfa22e228d4e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.142326 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.144800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.144800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.144800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.144800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.145739 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.156595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1371/2748] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:51.160703 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.170847 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4beaed-3100-4b64-a357-2334b1cc5ecb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.175737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.176312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.176594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.176873 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.177494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.179108 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.189314 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1372/2749] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:51.193167 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.202557 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b58d0-5830-4916-b120-9b550d9b40be admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.207904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf3494-3eb6-4c73-8158-d90d0f007bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.207904 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3494-3eb6-4c73-8158-d90d0f007bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.208187 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3494-3eb6-4c73-8158-d90d0f007bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.208313 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3494-3eb6-4c73-8158-d90d0f007bf7 admin admin] PATH_INFO: `/v3/projects/8db48b81ba08481fb7cd350fc998b3a7/users/f7ba68824a2044148b6f1224f576c6e5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.222817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf3494-3eb6-4c73-8158-d90d0f007bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db48b81ba08481fb7cd350fc998b3a7, user_id=f7ba68824a2044148b6f1224f576c6e5, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.225468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf3494-3eb6-4c73-8158-d90d0f007bf7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.264055 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be3656e855843d286f552b3d3a422da'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.264722 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be3656e855843d286f552b3d3a422da'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.277139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be3656e855843d286f552b3d3a422da'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.284057 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be3656e855843d286f552b3d3a422da'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.313477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1373/2750] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:51 2022] PUT /identity/v3/projects/8db48b81ba08481fb7cd350fc998b3a7/users/f7ba68824a2044148b6f1224f576c6e5/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:51.328094 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.328479 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.328755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.356775 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] MFA Rules not processed for user `f7ba68824a2044148b6f1224f576c6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:51.358010 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:51.370927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1014d9fa-c572-4f46-864f-d0915a213d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be3656e855843d286f552b3d3a422da'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.377570 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1378/2751] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:51 2022] DELETE /identity/v3/users/5be3656e855843d286f552b3d3a422da => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:51.381985 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.382233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:51.382445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.382724 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.403777 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:51.407540 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:51.424120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1374/2752] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:51.435981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.437431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:51.466367 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1379/2753] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:51.475819 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.477175 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:51.497619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318de1-becc-459f-a854-c6b2c0a7e30a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.535697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ad6b1-052b-498a-873e-497600482da7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.539336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.540193 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.540412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.540672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.549515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157e038-c7b3-40d7-96fc-b9a40ba5a72b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.549817 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.550121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157e038-c7b3-40d7-96fc-b9a40ba5a72b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.550425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157e038-c7b3-40d7-96fc-b9a40ba5a72b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.550694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157e038-c7b3-40d7-96fc-b9a40ba5a72b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.551289 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.560672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1380/2754] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:51.608877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157e038-c7b3-40d7-96fc-b9a40ba5a72b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.610359 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157e038-c7b3-40d7-96fc-b9a40ba5a72b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.619263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1375/2755] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:51.645773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:51.647212 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:51.661070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260330f6-d666-41ed-a843-91431c4b8e93 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:51.666050 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:51.666703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.666988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.670807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] PATH_INFO: `/v3/users/f7ba68824a2044148b6f1224f576c6e5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.670807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ba68824a2044148b6f1224f576c6e5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:51.670807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:51.684593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:51.852625 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ba68824a2044148b6f1224f576c6e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.853480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ba68824a2044148b6f1224f576c6e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.866483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ba68824a2044148b6f1224f576c6e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.874051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ba68824a2044148b6f1224f576c6e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.963940 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ba68824a2044148b6f1224f576c6e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:51.970396 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1381/2756] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:51 2022] DELETE /identity/v3/users/f7ba68824a2044148b6f1224f576c6e5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:51.975007 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:51.975505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:51.975621 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:51.997792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:51.997792 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:52.003295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:52.005304 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:52.031263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:52.063179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1376/2757] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:52.073262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:52.074754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:52.079831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525d48f-0f97-4ae7-abb8-9b5d2e16e48a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:52.084646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:52.085281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:52.085562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:52.085864 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] PATH_INFO: `/v3/projects/9e65a294c74f40e390c1f7832f7e8a64` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:52.087316 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e65a294c74f40e390c1f7832f7e8a64)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:52.089297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:52.131967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7cb07a-f537-4390-a286-e5d23886f937 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:52.135688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:52.136311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:52.136573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:52.136857 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:52.145701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:52.147216 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:52.156729 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1377/2758] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:52.201618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e65a294c74f40e390c1f7832f7e8a64', 'request_id': 'req-e0f78203-be83-4fea-9d3d-e118c65dd0b6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.308363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e65a294c74f40e390c1f7832f7e8a64', 'request_id': 'req-e0f78203-be83-4fea-9d3d-e118c65dd0b6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.314765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e65a294c74f40e390c1f7832f7e8a64', 'request_id': 'req-e0f78203-be83-4fea-9d3d-e118c65dd0b6'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.326045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] The token cache is being invalidate because project 9e65a294c74f40e390c1f7832f7e8a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:52.326354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.327003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.328503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1382/2759] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:52 2022] DELETE /identity/v3/projects/9e65a294c74f40e390c1f7832f7e8a64 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:52.332730 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:52.343155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d341b-ba32-4ba9-80d2-f7c6a71ddb8b service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:52.408155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:52.408791 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:52.409083 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:52.409464 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] PATH_INFO: `/v3/users/8107587901bd4d34bc0b95621b9248a7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:52.415450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8107587901bd4d34bc0b95621b9248a7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:52.415450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:52.601216 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:52.603778 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:52.611512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8107587901bd4d34bc0b95621b9248a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.618039 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f78203-be83-4fea-9d3d-e118c65dd0b6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:52.623401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:52.624044 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:52.624323 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:52.624593 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] PATH_INFO: `/v3/projects/8db48b81ba08481fb7cd350fc998b3a7` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:52.625721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db48b81ba08481fb7cd350fc998b3a7)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:52.627769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:52.646113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:52.714085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8107587901bd4d34bc0b95621b9248a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.714767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8107587901bd4d34bc0b95621b9248a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.728896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8107587901bd4d34bc0b95621b9248a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.732368 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db48b81ba08481fb7cd350fc998b3a7', 'request_id': 'req-d185a9e2-b257-41c3-8d08-9b87e12baf98'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.736074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-496f3b6e-51e3-4efb-aea6-7225015ff666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8107587901bd4d34bc0b95621b9248a7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.737571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:52.742499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1378/2760] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:52 2022] DELETE /identity/v3/users/8107587901bd4d34bc0b95621b9248a7 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:52.804113 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db48b81ba08481fb7cd350fc998b3a7', 'request_id': 'req-d185a9e2-b257-41c3-8d08-9b87e12baf98'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.811229 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db48b81ba08481fb7cd350fc998b3a7', 'request_id': 'req-d185a9e2-b257-41c3-8d08-9b87e12baf98'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.823765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] The token cache is being invalidate because project 8db48b81ba08481fb7cd350fc998b3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:52.824067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.824717 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d185a9e2-b257-41c3-8d08-9b87e12baf98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:52.831033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1383/2761] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:52 2022] DELETE /identity/v3/projects/8db48b81ba08481fb7cd350fc998b3a7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:52.832444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:52.832919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:52.833047 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:52.975406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:52.976831 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.038003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1379/2762] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:53.102850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.103132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.103471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.122482 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:53.123683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.131284 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1384/2763] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:53.136060 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.145901 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360460a7-dde7-4b06-9114-10b8e07e4ae6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.151513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.152100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.152376 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.152681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.153365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:53.153612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.155233 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.162185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.164004 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.172518 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1380/2764] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:53.177447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.181568 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69256a-b4ac-481a-904f-a39f985d51e2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.186236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.187013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.187383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.187751 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] PATH_INFO: `/v3/projects/89bb1810d7eb4a63a9c69dd98268442b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.189015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89bb1810d7eb4a63a9c69dd98268442b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.189858 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d3dd-d7cb-44e9-8977-00cc123f35f2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.191305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.195440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.196040 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.197065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.197338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.197974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.200052 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.291385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1381/2765] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:53 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:53.295591 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.302591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89bb1810d7eb4a63a9c69dd98268442b', 'request_id': 'req-aca0fbc5-379f-472d-90ef-49b42a81fbd2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.356964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9261f0-df66-4924-80b8-da46977e9e87 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.361657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.362266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.362595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.362862 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.363512 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.365409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.370461 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89bb1810d7eb4a63a9c69dd98268442b', 'request_id': 'req-aca0fbc5-379f-472d-90ef-49b42a81fbd2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.376943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89bb1810d7eb4a63a9c69dd98268442b', 'request_id': 'req-aca0fbc5-379f-472d-90ef-49b42a81fbd2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.389640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] The token cache is being invalidate because project 89bb1810d7eb4a63a9c69dd98268442b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:53.389935 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.390628 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.392177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1385/2766] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:53 2022] DELETE /identity/v3/projects/89bb1810d7eb4a63a9c69dd98268442b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:53.395941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.397266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.452327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1382/2767] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:53 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:53.456478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.458139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.464890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0fbc5-379f-472d-90ef-49b42a81fbd2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.469660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.470295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.470600 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e7b3f-8666-4692-a02d-b21cfb2fd143 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.470730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.470942 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] PATH_INFO: `/v3/users/f09981d845294c47970ba99f74bb784b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.472102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f09981d845294c47970ba99f74bb784b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.474025 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.475605 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.476218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.476501 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.476770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.477420 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.479287 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.490456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1383/2768] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:53.495276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.500825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:53.505224 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ab5d1-b341-4a14-b793-ff7cca4b96b3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.510745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.511341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.511644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.511914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.512675 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.514268 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.524271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1384/2769] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:53.528530 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.538239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a5df1-729b-4fc6-9f10-61a535d8b91a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.543737 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea47b4b-5e40-4229-ab8e-ed2a1da6f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.544365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea47b4b-5e40-4229-ab8e-ed2a1da6f118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.544649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea47b4b-5e40-4229-ab8e-ed2a1da6f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.544924 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea47b4b-5e40-4229-ab8e-ed2a1da6f118 admin admin] PATH_INFO: `/v3/projects/a6e5458471854fc8ab13b44366a2c919/users/752edebef19f4012a2a4d1b45a57b263/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.559655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea47b4b-5e40-4229-ab8e-ed2a1da6f118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e5458471854fc8ab13b44366a2c919, user_id=752edebef19f4012a2a4d1b45a57b263, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.562267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea47b4b-5e40-4229-ab8e-ed2a1da6f118 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.634143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1385/2770] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:53 2022] PUT /identity/v3/projects/a6e5458471854fc8ab13b44366a2c919/users/752edebef19f4012a2a4d1b45a57b263/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:53.644143 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.644431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.644702 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.657614 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:53.669647 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f09981d845294c47970ba99f74bb784b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.670391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f09981d845294c47970ba99f74bb784b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.674311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] MFA Rules not processed for user `752edebef19f4012a2a4d1b45a57b263`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:53.675569 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.683043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f09981d845294c47970ba99f74bb784b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.691315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f09981d845294c47970ba99f74bb784b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.740090 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1386/2771] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:53.752012 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.753411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.753883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f09981d845294c47970ba99f74bb784b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:53.760809 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1386/2772] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:53 2022] DELETE /identity/v3/users/f09981d845294c47970ba99f74bb784b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:53.813299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c267194-695e-46b1-809b-a4ce25715ac8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:53.869024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:53.869638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:53.869919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:53.870192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:53.872021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:53.932107 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:53.933560 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:53.943049 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1387/2773] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:53.969035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:53.970481 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.045616 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f6a52-aea0-4988-856b-1364782d4b37 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:54.048858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:54.049491 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.049769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.050042 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] PATH_INFO: `/v3/users/752edebef19f4012a2a4d1b45a57b263` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.051200 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=752edebef19f4012a2a4d1b45a57b263)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:54.053142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:54.224000 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:54.225457 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.239947 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de33e2-6a60-4ca3-9d82-b9c6483b1c3e service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:54.245720 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:54.246550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.246676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.246918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] PATH_INFO: `/v3/projects/487fe37416e14ad0b66a166022a87a79` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.248093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=487fe37416e14ad0b66a166022a87a79)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:54.250102 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:54.251754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:54.278082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752edebef19f4012a2a4d1b45a57b263'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.278683 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752edebef19f4012a2a4d1b45a57b263'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.291318 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752edebef19f4012a2a4d1b45a57b263'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.298649 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752edebef19f4012a2a4d1b45a57b263'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.393859 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487fe37416e14ad0b66a166022a87a79', 'request_id': 'req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.438498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a254e3f8-4d72-4be4-94a5-06d29f12bc92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752edebef19f4012a2a4d1b45a57b263'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.444883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1387/2774] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:53 2022] DELETE /identity/v3/users/752edebef19f4012a2a4d1b45a57b263 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:54.449270 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.449558 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.449834 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.468433 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:54.474804 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.483354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:54.528622 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1388/2775] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:54.538422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:54.540806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.576954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487fe37416e14ad0b66a166022a87a79', 'request_id': 'req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.583350 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487fe37416e14ad0b66a166022a87a79', 'request_id': 'req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.595312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] The token cache is being invalidate because project 487fe37416e14ad0b66a166022a87a79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:54.595678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.596399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c9d9d4ad-9580-4ad0-82f2-acc8d50d6cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:54.598170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1388/2776] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:54 2022] DELETE /identity/v3/projects/487fe37416e14ad0b66a166022a87a79 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:41:54.598306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb2d26-a49e-4bf3-9ccb-9e7d4ab97092 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:54.601791 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:54.602847 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.603099 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.603373 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.603793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.604043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.604312 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.605111 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.617702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:54.619290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c8ca9-8460-46e4-a16a-10575d9cbfaa service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:54.624061 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:54.625264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.629805 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1389/2777] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:41:54.638204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1389/2778] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:54.644086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.644483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.644759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.669527 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:54.671129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:54.682772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1390/2779] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:54.683663 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:54.693294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc612a-6308-48aa-9b41-f6e652ea7ee6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:54.698710 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:54.699322 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.699603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.699877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.700528 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:54.700828 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:54.702507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:54.715148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1390/2780] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:54.719576 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:54.730224 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c0d94-22fd-4215-8ecb-5d41a83b82b7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:54.735241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:54.736578 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.736578 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.736578 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.737180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:54.738984 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:54.832201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1391/2781] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:54 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:54.836001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:54.847893 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12562cc-ef87-40cb-8282-dc733139d89d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:54.910168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:54.910928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:54.911084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:54.911341 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:54.911982 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:54.913903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.035087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1391/2782] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:54 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:55.039539 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.061035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33382-0b3e-4cc0-8ae9-a59d8a601512 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.061035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.061035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.061035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.061502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.061502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.061502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.068502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1392/2783] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:55.072418 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.082419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdea85-a7b2-4de8-aa1d-9ea0a87f5270 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.088789 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.091658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.091658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.091658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.091658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.092356 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.093282 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.095603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:55.110402 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05523058-1599-4f38-80c0-1fa552569d5f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.114896 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1392/2784] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:55.121180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.121180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.121180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.121180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] PATH_INFO: `/v3/projects/a6e5458471854fc8ab13b44366a2c919` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.121969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e5458471854fc8ab13b44366a2c919)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.121969 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.122763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.132770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dad749-c850-4da4-abfd-ef09c0ed1586 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.137677 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b069cc7-a3c7-4498-ba75-687febfb3ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.138447 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069cc7-a3c7-4498-ba75-687febfb3ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.138788 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069cc7-a3c7-4498-ba75-687febfb3ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.139125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069cc7-a3c7-4498-ba75-687febfb3ebb admin admin] PATH_INFO: `/v3/projects/4f413d9d76a94a2cbf4ffe4b877b9841/users/ce3d22bff7624710a6499727b7fd51a8/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.154025 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b069cc7-a3c7-4498-ba75-687febfb3ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f413d9d76a94a2cbf4ffe4b877b9841, user_id=ce3d22bff7624710a6499727b7fd51a8, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.156704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b069cc7-a3c7-4498-ba75-687febfb3ebb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.215433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1393/2785] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:55 2022] PUT /identity/v3/projects/4f413d9d76a94a2cbf4ffe4b877b9841/users/ce3d22bff7624710a6499727b7fd51a8/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:55.226064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e5458471854fc8ab13b44366a2c919', 'request_id': 'req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:55.226622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.227065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.227171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.254797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] MFA Rules not processed for user `ce3d22bff7624710a6499727b7fd51a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:55.256001 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:55.267475 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:55.289546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e5458471854fc8ab13b44366a2c919', 'request_id': 'req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:55.296454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e5458471854fc8ab13b44366a2c919', 'request_id': 'req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:55.308783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] The token cache is being invalidate because project a6e5458471854fc8ab13b44366a2c919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:55.309072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:55.309711 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:55.311306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1393/2786] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:55 2022] DELETE /identity/v3/projects/a6e5458471854fc8ab13b44366a2c919 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:55.324221 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1394/2787] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:41:55.335959 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.337502 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:55.348107 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:55.410936 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e57a75-e54f-4ae5-b3aa-c54d3cb88047 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.416700 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.417336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.417619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.417890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.418591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.420534 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.525890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1394/2788] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:55 2022] POST /identity/v3/projects => generated 389 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:55.527239 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.539152 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2f8e7-b254-4256-a24b-f5a342c5916d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.595754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.596276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.596553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.596824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.597456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.599394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.678185 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1395/2789] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:55 2022] POST /identity/v3/users => generated 301 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:55.682946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.693423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae22e30-5410-4c4a-846c-6a2b7ace1ef8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.698948 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.699549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.699844 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.700115 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.700756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.702356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.712756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1395/2790] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:55.720075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.730113 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353e0e5-38c9-4c3b-8ef2-08ae4e2abf4d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.734696 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.735375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.735658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.735937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] PATH_INFO: `/v3/projects/6bce9fd486804ea199f30303cb47ee25/users/ffac75044efa40c6bfd4f509f639d8ba/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.750823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bce9fd486804ea199f30303cb47ee25, user_id=ffac75044efa40c6bfd4f509f639d8ba, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.753467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.829721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1396/2791] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:55 2022] PUT /identity/v3/projects/6bce9fd486804ea199f30303cb47ee25/users/ffac75044efa40c6bfd4f509f639d8ba/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:55.833709 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.843747 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5323687-3ac7-4630-9fd0-8d0228a831d9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.900857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc56649-49bc-4471-b2c0-e837931d743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.901387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc56649-49bc-4471-b2c0-e837931d743f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.901666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc56649-49bc-4471-b2c0-e837931d743f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.901934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc56649-49bc-4471-b2c0-e837931d743f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.902607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc56649-49bc-4471-b2c0-e837931d743f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.904195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc56649-49bc-4471-b2c0-e837931d743f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.914194 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1396/2792] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:55.918977 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:55.929223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738402a-874c-4575-8818-a3f648ea2ebd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:55.934723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:55.935373 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.935659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.935929 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] PATH_INFO: `/v3/projects/6bce9fd486804ea199f30303cb47ee25/users/ffac75044efa40c6bfd4f509f639d8ba/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.937842 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bce9fd486804ea199f30303cb47ee25, user_id=ffac75044efa40c6bfd4f509f639d8ba, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:55.940496 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:55.951428 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1397/2793] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:55 2022] PUT /identity/v3/projects/6bce9fd486804ea199f30303cb47ee25/users/ffac75044efa40c6bfd4f509f639d8ba/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:55.962590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:55.962875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:55.963148 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:55.991302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] MFA Rules not processed for user `ffac75044efa40c6bfd4f509f639d8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:55.992517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:56.060288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1397/2794] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:55 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:56.071139 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:56.072815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:56.128724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b6e70-359e-4ae5-b656-5fa422d57bcb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:56.132361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e3bcd-d061-4e2e-af28-63b9b7ceb95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:56.132959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3bcd-d061-4e2e-af28-63b9b7ceb95d service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:56.133241 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3bcd-d061-4e2e-af28-63b9b7ceb95d service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:56.133682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e3bcd-d061-4e2e-af28-63b9b7ceb95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:56.142705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e3bcd-d061-4e2e-af28-63b9b7ceb95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:56.144136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e3bcd-d061-4e2e-af28-63b9b7ceb95d service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:56.153936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1398/2795] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:56.288215 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:56.293020 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:56.351267 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786390a4-3909-4b04-8af4-7f612252f06a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:56.356514 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:56.357137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:56.357413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:56.357682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] PATH_INFO: `/v3/users/ce3d22bff7624710a6499727b7fd51a8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:56.358882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce3d22bff7624710a6499727b7fd51a8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:56.360783 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:56.600979 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce3d22bff7624710a6499727b7fd51a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:56.601654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce3d22bff7624710a6499727b7fd51a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:56.613760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce3d22bff7624710a6499727b7fd51a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:56.621526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce3d22bff7624710a6499727b7fd51a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:56.689291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce3d22bff7624710a6499727b7fd51a8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:56.695460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1398/2796] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:56 2022] DELETE /identity/v3/users/ce3d22bff7624710a6499727b7fd51a8 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:56.700580 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:56.701017 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:56.701155 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:56.721013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:56.722213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:56.726050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:56.781755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1399/2797] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:56.791691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:56.793089 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:56.851315 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e924a-03bd-4290-b4bd-7a8c49325883 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:56.855061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:56.855669 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:56.855949 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:56.856221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:56.866045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:56.867515 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:56.877156 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1399/2798] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:41:57.303648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:57.305766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:57.319581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19371e16-2ed5-4091-b16d-40124c52db16 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:57.325799 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:57.326690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:57.326970 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:57.327237 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] PATH_INFO: `/v3/projects/4f413d9d76a94a2cbf4ffe4b877b9841` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:57.328399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f413d9d76a94a2cbf4ffe4b877b9841)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:57.330463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:57.443824 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f413d9d76a94a2cbf4ffe4b877b9841', 'request_id': 'req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.507781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f413d9d76a94a2cbf4ffe4b877b9841', 'request_id': 'req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.514209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f413d9d76a94a2cbf4ffe4b877b9841', 'request_id': 'req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.525997 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] The token cache is being invalidate because project 4f413d9d76a94a2cbf4ffe4b877b9841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:57.526279 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.526996 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.528521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1400/2799] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:57 2022] DELETE /identity/v3/projects/4f413d9d76a94a2cbf4ffe4b877b9841 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:57.532472 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:57.542459 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f0e13-78a3-4136-9e5c-6c38f9e1127a service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:57.562546 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:57.599253 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:57.599872 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:57.600149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:57.600417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] PATH_INFO: `/v3/users/ffac75044efa40c6bfd4f509f639d8ba` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:57.601560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffac75044efa40c6bfd4f509f639d8ba)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:57.603861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:57.794748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffac75044efa40c6bfd4f509f639d8ba'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.795411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffac75044efa40c6bfd4f509f639d8ba'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.808843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffac75044efa40c6bfd4f509f639d8ba'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.815858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffac75044efa40c6bfd4f509f639d8ba'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.885832 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-594c1d20-b7dc-4247-b4db-6659d56c7e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffac75044efa40c6bfd4f509f639d8ba'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:57.892272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1400/2800] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:41:57 2022] DELETE /identity/v3/users/ffac75044efa40c6bfd4f509f639d8ba => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:58.281299 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:58.282823 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:58.296583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a2d5-9f9e-4cf8-b845-a92e19e0458a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:58.358648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:58.359288 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:58.359572 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:58.359846 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] PATH_INFO: `/v3/projects/6bce9fd486804ea199f30303cb47ee25` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:58.361050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bce9fd486804ea199f30303cb47ee25)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:58.363136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:58.471128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:58.480808 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bce9fd486804ea199f30303cb47ee25', 'request_id': 'req-2eec9961-105c-4706-9456-8c5a71d8f6c3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:58.530450 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bce9fd486804ea199f30303cb47ee25', 'request_id': 'req-2eec9961-105c-4706-9456-8c5a71d8f6c3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:58.536331 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bce9fd486804ea199f30303cb47ee25', 'request_id': 'req-2eec9961-105c-4706-9456-8c5a71d8f6c3'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:58.548033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] The token cache is being invalidate because project 6bce9fd486804ea199f30303cb47ee25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:41:58.548316 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:58.548935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2eec9961-105c-4706-9456-8c5a71d8f6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:41:58.550431 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1401/2801] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:41:58 2022] DELETE /identity/v3/projects/6bce9fd486804ea199f30303cb47ee25 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:58.555583 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:58.555905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:58.556181 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:58.575617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:58.576750 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:58.634494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1401/2802] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:58.642332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:58.642651 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:58.642925 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:58.663192 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:58.664383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:58.672871 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1402/2803] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:58.676967 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:58.686756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042f5d1-2d87-420b-bbfa-30511ef628f4 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:58.691278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:58.691865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:58.692142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:58.692413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:58.693053 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:41:58.693348 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:58.694963 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:58.706250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1402/2804] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:41:58.710319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:58.719787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a2207-f6f1-414d-a938-4840b83f6839 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:58.724332 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:58.724916 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:58.725191 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:58.725454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:58.726077 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:58.728633 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:58.828949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1403/2805] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:41:58 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:41:58.832995 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:58.844072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ab2b8-4ea7-463d-bd1d-88a6bc5f1552 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:58.872873 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:41:58.901725 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:58.902369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:58.902796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:58.903423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:58.904119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:58.906038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:58.987640 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1403/2806] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:41:58 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:41:58.991979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:59.002571 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fe88c-a903-4e20-9190-8c46f7d42489 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:59.007116 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:59.007736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:59.008125 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:59.008882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:59.009547 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:59.011276 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:59.021151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1404/2807] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:41:59.025624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:59.035856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f29e-bb5a-46d9-83d8-a5b94253fa4f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:59.040761 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c5312-42c1-4e6b-a20a-8b3f3df1f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:59.041356 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c5312-42c1-4e6b-a20a-8b3f3df1f243 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:59.041668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c5312-42c1-4e6b-a20a-8b3f3df1f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:59.041945 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c5312-42c1-4e6b-a20a-8b3f3df1f243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:59.042618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c5312-42c1-4e6b-a20a-8b3f3df1f243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:59.044213 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c5312-42c1-4e6b-a20a-8b3f3df1f243 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:59.054300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1404/2808] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 03 01:41:59.058385 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:59.068054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65387269-ed82-4f2c-ad08-70d6c96cce15 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:59.073289 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:59.073927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:59.074204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:59.074510 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] PATH_INFO: `/v3/projects/c0af83b822a54e8fbb23531d72bbde2e/users/15f21ab07d294e04936d3471e5d52766/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:59.089189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0af83b822a54e8fbb23531d72bbde2e, user_id=15f21ab07d294e04936d3471e5d52766, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:59.091802 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:59.188531 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1405/2809] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:41:59 2022] PUT /identity/v3/projects/c0af83b822a54e8fbb23531d72bbde2e/users/15f21ab07d294e04936d3471e5d52766/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:41:59.199591 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:59.199875 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:59.200150 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:59.228454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] MFA Rules not processed for user `15f21ab07d294e04936d3471e5d52766`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:41:59.229668 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:59.292490 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1405/2810] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:41:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:41:59.303368 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:41:59.305120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:41:59.363840 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2cb51-caab-4bf5-b3cf-d656fff2bd8a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:41:59.367617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27b300-814e-45af-8b29-23890acaa507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:41:59.368771 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27b300-814e-45af-8b29-23890acaa507 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:41:59.369054 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27b300-814e-45af-8b29-23890acaa507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:41:59.369375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27b300-814e-45af-8b29-23890acaa507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:41:59.377685 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27b300-814e-45af-8b29-23890acaa507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:41:59.379169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27b300-814e-45af-8b29-23890acaa507 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:41:59.388625 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1406/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:41:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:09.290162 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:09.383934 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:09.384636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:09.449612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6a468-fcab-42f8-ae1c-b8219eb3f1ea None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:09.454305 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:09.454929 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:09.455211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:09.455510 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] PATH_INFO: `/v3/users/15f21ab07d294e04936d3471e5d52766` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:09.456704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f21ab07d294e04936d3471e5d52766)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:09.458697 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:09.654121 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f21ab07d294e04936d3471e5d52766'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:09.654789 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f21ab07d294e04936d3471e5d52766'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:09.668617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f21ab07d294e04936d3471e5d52766'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:09.676152 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f21ab07d294e04936d3471e5d52766'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:09.734975 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f21ab07d294e04936d3471e5d52766'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:09.741487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1406/2812] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:42:09 2022] DELETE /identity/v3/users/15f21ab07d294e04936d3471e5d52766 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:09.746119 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:09.746437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:09.746721 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:09.765440 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:09.769945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:09.826232 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1407/2813] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:09.836745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:09.838528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:09.896228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34cb5c-62bc-4d10-a869-9077f03c88b4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:09.900624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384ab5e-217f-41de-9142-18b79c9cd321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:09.901232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384ab5e-217f-41de-9142-18b79c9cd321 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:09.901513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384ab5e-217f-41de-9142-18b79c9cd321 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:09.901782 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384ab5e-217f-41de-9142-18b79c9cd321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:09.911206 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384ab5e-217f-41de-9142-18b79c9cd321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:09.912579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384ab5e-217f-41de-9142-18b79c9cd321 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:09.922905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1407/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:10.436452 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:10.439629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:10.452228 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49f5d6-a8e9-4606-9f8b-e84c3191683e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:10.457338 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:10.458037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:10.458375 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:10.458795 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] PATH_INFO: `/v3/projects/c0af83b822a54e8fbb23531d72bbde2e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:10.459886 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0af83b822a54e8fbb23531d72bbde2e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:10.462136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:10.566744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0af83b822a54e8fbb23531d72bbde2e', 'request_id': 'req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:10.639945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0af83b822a54e8fbb23531d72bbde2e', 'request_id': 'req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:10.646575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0af83b822a54e8fbb23531d72bbde2e', 'request_id': 'req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:10.659264 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] The token cache is being invalidate because project c0af83b822a54e8fbb23531d72bbde2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:42:10.659554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:10.660456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b7dfdee7-8c6f-45f6-b8ba-bbf8484aa16d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:10.662014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1408/2815] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:42:10 2022] DELETE /identity/v3/projects/c0af83b822a54e8fbb23531d72bbde2e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:10.667028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:10.667547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:10.667655 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:10.687547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:10.688685 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:10.706256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:10.749850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1408/2816] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:42:10.760150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:10.760495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:10.760786 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:10.782087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:10.783303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:10.792655 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1409/2817] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:10.796911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:10.809125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcb08b-be5b-4889-bd94-1fdfa1078033 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:10.814266 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:10.815001 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:10.815311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:10.815620 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:10.816365 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:42:10.816703 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:10.818572 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:10.832758 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1409/2818] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:42:10.837020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:10.846141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab94632-a00b-4b63-8af5-3ff4a1ed800c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:10.850806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:10.851395 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:10.851672 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:10.852455 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:10.853080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:10.854994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:10.943973 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1410/2819] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:42:10 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:10.948422 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:10.958400 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8e231-3c5c-4523-b8b6-c364ac216d6f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:11.023528 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:11.024987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:11.025345 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:11.025691 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:11.026610 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:11.029005 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:11.107696 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1410/2820] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:42:10 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:42:11.111958 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:11.122159 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe321a7-956c-48fb-beea-8cddae2ae2a4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:11.127004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:11.127683 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:11.127975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:11.128259 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:11.128969 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:11.130832 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:11.142682 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1411/2821] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:11.146741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:11.158278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc6c00-da7b-420c-9f89-241ebe54c299 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:11.163524 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc0456-aef0-4f15-82c6-1d0f49eed38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:11.164163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc0456-aef0-4f15-82c6-1d0f49eed38b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:11.164605 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc0456-aef0-4f15-82c6-1d0f49eed38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:11.164877 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc0456-aef0-4f15-82c6-1d0f49eed38b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:11.165516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc0456-aef0-4f15-82c6-1d0f49eed38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:11.167285 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc0456-aef0-4f15-82c6-1d0f49eed38b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:11.178611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1411/2822] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:11.183499 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:11.193399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f018e20-4fd9-4d08-bdb2-de9b73e04f8b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:11.198743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:11.199324 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:11.199604 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:11.199874 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] PATH_INFO: `/v3/projects/ddbd6e25b1354e10a298b39bd110a1b7/users/38596d6d07364db6898249f4e55c1404/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:11.213994 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbd6e25b1354e10a298b39bd110a1b7, user_id=38596d6d07364db6898249f4e55c1404, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:11.216658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:11.302590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1412/2823] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:42:11 2022] PUT /identity/v3/projects/ddbd6e25b1354e10a298b39bd110a1b7/users/38596d6d07364db6898249f4e55c1404/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:11.313350 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:11.313679 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:11.313939 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:11.342547 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] MFA Rules not processed for user `38596d6d07364db6898249f4e55c1404`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:11.343767 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:11.408172 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1412/2824] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:42:11.419612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:11.421294 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:11.442975 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:11.477213 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f5289-7327-40ab-b5a4-18fd423faf25 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:11.480755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104caaad-bfbe-437b-8223-b2acd3263cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:11.481355 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104caaad-bfbe-437b-8223-b2acd3263cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:11.481657 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104caaad-bfbe-437b-8223-b2acd3263cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:11.481931 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104caaad-bfbe-437b-8223-b2acd3263cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:11.490603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104caaad-bfbe-437b-8223-b2acd3263cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:11.492074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104caaad-bfbe-437b-8223-b2acd3263cb4 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:11.501095 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1413/2825] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:21.404947 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:21.547758 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=17 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:34.524048 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:34.586012 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f9ab0-fd7a-4d21-b76a-a6ca42f2c026 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:34.590542 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:34.591167 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:34.591448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:34.591721 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] PATH_INFO: `/v3/users/38596d6d07364db6898249f4e55c1404` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:34.592826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=38596d6d07364db6898249f4e55c1404)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:34.594775 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:34.753474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38596d6d07364db6898249f4e55c1404'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:34.754125 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38596d6d07364db6898249f4e55c1404'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:34.766180 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38596d6d07364db6898249f4e55c1404'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:34.777833 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38596d6d07364db6898249f4e55c1404'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:34.879413 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38596d6d07364db6898249f4e55c1404'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:34.886744 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1413/2826] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:42:34 2022] DELETE /identity/v3/users/38596d6d07364db6898249f4e55c1404 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:34.891170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:34.891658 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:34.891768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:34.911772 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:34.913173 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:34.971209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1414/2827] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:34.981187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:34.983077 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:35.045465 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dbc98-c56a-4d95-9420-7f690628a63c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:35.049102 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3eea-0416-4885-ac35-31b6fdb2d6d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:35.049704 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3eea-0416-4885-ac35-31b6fdb2d6d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:35.049982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3eea-0416-4885-ac35-31b6fdb2d6d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:35.050250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3eea-0416-4885-ac35-31b6fdb2d6d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:35.059483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3eea-0416-4885-ac35-31b6fdb2d6d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:35.060904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d3eea-0416-4885-ac35-31b6fdb2d6d5 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:35.070774 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1414/2828] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:35.547639 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:35.549147 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:35.561990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139674ea-a6fd-4287-a1c5-3c9603e03167 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:35.566678 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:35.569013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:35.569013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:35.569013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] PATH_INFO: `/v3/projects/ddbd6e25b1354e10a298b39bd110a1b7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:35.569013 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbd6e25b1354e10a298b39bd110a1b7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:35.571204 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:35.737425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbd6e25b1354e10a298b39bd110a1b7', 'request_id': 'req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:35.833340 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbd6e25b1354e10a298b39bd110a1b7', 'request_id': 'req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:35.839822 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbd6e25b1354e10a298b39bd110a1b7', 'request_id': 'req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:35.851516 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] The token cache is being invalidate because project ddbd6e25b1354e10a298b39bd110a1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:42:35.854404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:35.854404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b91c9a81-60f1-476b-8bf6-f4cf15c103ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:35.854404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1415/2829] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:42:35 2022] DELETE /identity/v3/projects/ddbd6e25b1354e10a298b39bd110a1b7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:35.859618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:35.860112 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:35.860214 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:35.886588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:35.887862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:35.905728 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:35.950028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1415/2830] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:35.957855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:35.960085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:35.960085 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:35.979502 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:35.980724 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:35.988740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1416/2831] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:35.993090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.002666 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea1d62-bf0a-4668-9e22-25f3c9628c42 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.007211 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.008071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.008590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.009047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.010073 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:42:36.010411 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.012038 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.024117 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1416/2832] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:42:36.028304 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.037767 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6696afc-fa54-4c37-aa92-fbf9390b41a8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.043363 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.043964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.044244 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.044514 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.045151 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.047120 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.137227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1417/2833] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:42:36 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:36.141363 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.152132 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1daee0-8166-48fe-8fd3-036027d5175b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.208262 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.208857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.209139 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.209410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.210063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.212660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.330994 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1417/2834] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:42:36 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:36.335154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.345942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae73ab4-604c-4eb5-b9ae-1594e2f4020e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.350529 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.351117 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.351694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.351984 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.352620 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.354208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.363951 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1418/2835] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:36.368085 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.378008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36346b5-e41b-4fc9-a949-20f8717dd366 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.383689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b8616-685e-4abe-bfdc-0762c171e725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.384301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b8616-685e-4abe-bfdc-0762c171e725 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.384612 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b8616-685e-4abe-bfdc-0762c171e725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.384835 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b8616-685e-4abe-bfdc-0762c171e725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.385474 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b8616-685e-4abe-bfdc-0762c171e725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.387101 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b8616-685e-4abe-bfdc-0762c171e725 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.398730 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1418/2836] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:36.402634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.413353 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcc875-daa8-4c8b-bb62-13f86f07f862 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.418078 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.418839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.419121 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.419397 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] PATH_INFO: `/v3/projects/347e480a70d549b08ece0b6958e885d4/users/8a8ae2d9ccca4eedbbc89c765c8cf664/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.433736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] RBAC: Authorizing `identity:create_grant(project_id=347e480a70d549b08ece0b6958e885d4, user_id=8a8ae2d9ccca4eedbbc89c765c8cf664, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.436425 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.537631 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1419/2837] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:42:36 2022] PUT /identity/v3/projects/347e480a70d549b08ece0b6958e885d4/users/8a8ae2d9ccca4eedbbc89c765c8cf664/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:36.617708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.618092 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.618408 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.646354 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] MFA Rules not processed for user `8a8ae2d9ccca4eedbbc89c765c8cf664`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:36.647577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:36.710887 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1419/2838] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:36.715280 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.735967 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:36.776404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c4fc3-ca94-4316-9fc5-9404b63d46de admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.780990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.781594 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.781876 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.782150 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.782878 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.784779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:36.920394 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1420/2839] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:42:36 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:36.924483 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:36.936552 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e269-08be-4cc4-95e5-cd1c4ac966b3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:36.951059 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:36.994786 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:36.995392 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:36.995672 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:36.995943 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:36.996577 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:36.998599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.067395 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1420/2840] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:42:36 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:42:37.071603 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.082009 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d85a2-af4e-4c6e-bea0-8756939242b4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.086649 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.087245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.087526 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.087797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.088477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.090034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.100489 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1421/2841] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:37.105079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.115149 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d9fe7-7e92-4c0d-bf31-9fd40696acba admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.120425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fdae-ac52-43cc-a849-234b3d779efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.121008 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fdae-ac52-43cc-a849-234b3d779efa admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.121286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fdae-ac52-43cc-a849-234b3d779efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.121553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fdae-ac52-43cc-a849-234b3d779efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.122185 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211fdae-ac52-43cc-a849-234b3d779efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.123800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211fdae-ac52-43cc-a849-234b3d779efa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.133831 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1421/2842] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:37.137943 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.147851 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd486ae-d2bb-4831-9d6d-9165888f281b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.152766 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.153412 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.153690 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.153959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] PATH_INFO: `/v3/projects/eea85c4a025d413aa87da248e6bfe11f/users/394a02c801a14ec499e9eb6fdb1eda94/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.170238 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea85c4a025d413aa87da248e6bfe11f, user_id=394a02c801a14ec499e9eb6fdb1eda94, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.173034 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.255093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1422/2843] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:42:37 2022] PUT /identity/v3/projects/eea85c4a025d413aa87da248e6bfe11f/users/394a02c801a14ec499e9eb6fdb1eda94/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:42:37.265486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.265773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.266043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.294678 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] MFA Rules not processed for user `394a02c801a14ec499e9eb6fdb1eda94`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:37.295897 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:37.358759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1422/2844] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:37.363474 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.424426 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553cf7c-fe45-4373-8830-ebf3749ed4fc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.428895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.429508 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.429885 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.430158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.430817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.432709 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.595433 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1423/2845] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:42:37 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:37.599427 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.610735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55161b3-a797-4a87-ba2b-c304e28c263a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.667650 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.668349 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.668633 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.668906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.669562 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.671533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.770741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1423/2846] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:42:37 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:37.775463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.785959 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260146c-59ed-4c18-9ed6-62affbf48884 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.790908 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.793852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.793852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.793852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.793852 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.794267 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.805328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1424/2847] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:37.809586 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.821425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314df5-7f6f-4144-8c68-34f80cdd7efe admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.826061 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:37.826745 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:37.827024 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:37.827296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] PATH_INFO: `/v3/projects/01177607dede4eacb4319e1c761232d4/users/72a56c5d22494235baa09ff4638d69b8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:37.841221 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:37.844376 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01177607dede4eacb4319e1c761232d4, user_id=72a56c5d22494235baa09ff4638d69b8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:37.847207 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:37.934769 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1424/2848] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:42:37 2022] PUT /identity/v3/projects/01177607dede4eacb4319e1c761232d4/users/72a56c5d22494235baa09ff4638d69b8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:42:37.938877 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:37.949595 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204acc4-b793-4ecd-a86a-bbb94c710e78 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:37.989839 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:38.004043 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d40a3a-cade-4c81-b6c8-9bfba6fcd5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:38.004637 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40a3a-cade-4c81-b6c8-9bfba6fcd5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:38.004913 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40a3a-cade-4c81-b6c8-9bfba6fcd5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:38.005180 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d40a3a-cade-4c81-b6c8-9bfba6fcd5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:38.005804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d40a3a-cade-4c81-b6c8-9bfba6fcd5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:38.007942 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d40a3a-cade-4c81-b6c8-9bfba6fcd5f2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:38.019903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1425/2849] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:38.021856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:38.032658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afbb33-533b-4387-9e28-c3d3e1ea6b8d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:38.037643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:38.038279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:38.038600 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:38.038860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] PATH_INFO: `/v3/projects/01177607dede4eacb4319e1c761232d4/users/72a56c5d22494235baa09ff4638d69b8/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:38.040907 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01177607dede4eacb4319e1c761232d4, user_id=72a56c5d22494235baa09ff4638d69b8, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:38.043689 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:38.053773 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1425/2850] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:42:38 2022] PUT /identity/v3/projects/01177607dede4eacb4319e1c761232d4/users/72a56c5d22494235baa09ff4638d69b8/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:38.065269 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:38.065554 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:38.065833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:38.093911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] MFA Rules not processed for user `72a56c5d22494235baa09ff4638d69b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:38.095156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:38.161365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1426/2851] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:38.172732 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:38.174423 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:38.231195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c5c49-362b-468e-b627-35be2af28f92 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:38.234889 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:38.235486 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:38.235765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:38.236033 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:38.292687 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:38.294045 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:38.302881 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1426/2852] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:42:40.394575 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:40.395990 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:40.408681 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb63874-99d9-4a13-a6a5-492737eba01d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:40.412494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:40.413089 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:40.413399 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:40.414432 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:40.475894 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:40.478768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:40.487918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1427/2853] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:41.921379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:41.930464 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d92c2-de68-43b7-896b-2c5e6beb0fed service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:41.934606 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:41.935278 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:41.935681 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:41.936035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:41.938052 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:41.975784 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:41.984646 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db876c-6a8d-443a-b530-a902674471e7 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:41.988387 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:41.988987 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:41.989271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:41.989570 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:41.998496 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:41.999940 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:42.001475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:42.002950 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6303-f9c0-4592-b1d3-d01bd1d3e74f service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:42.012352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1428/2854] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:42:42.012900 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1427/2855] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:47.720919 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:47.771262 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:47.783581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61880159-4b5b-419e-89dd-b4a8302761d3 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:47.788266 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:47.788887 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:47.789165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:47.789435 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] PATH_INFO: `/v3/users/8a8ae2d9ccca4eedbbc89c765c8cf664` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:47.790559 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8ae2d9ccca4eedbbc89c765c8cf664)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:47.792548 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:47.965494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8ae2d9ccca4eedbbc89c765c8cf664'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.039123 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8ae2d9ccca4eedbbc89c765c8cf664'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.039705 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8ae2d9ccca4eedbbc89c765c8cf664'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.052574 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8ae2d9ccca4eedbbc89c765c8cf664'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.059763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8ae2d9ccca4eedbbc89c765c8cf664'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.065761 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1429/2856] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:42:47 2022] DELETE /identity/v3/users/8a8ae2d9ccca4eedbbc89c765c8cf664 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:42:48.070500 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:48.071051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:48.071051 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:48.090286 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:48.093922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:48.117257 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:48.150560 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1428/2857] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:48.160722 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:48.162136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:48.217662 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785e2a-933c-4c7e-820d-27a6ed621ed8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:48.221298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:48.221892 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:48.222168 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:48.222467 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:48.231162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:48.232634 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:48.241918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1430/2858] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:48.673295 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:48.674797 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:48.687397 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34b09f-9dbd-4f34-84ea-1f0e866e4239 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:48.692242 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:48.692878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:48.693155 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:48.693439 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] PATH_INFO: `/v3/projects/347e480a70d549b08ece0b6958e885d4` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:48.694749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347e480a70d549b08ece0b6958e885d4)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:48.696530 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:48.807369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347e480a70d549b08ece0b6958e885d4', 'request_id': 'req-13683e78-5dad-4b91-971a-b17b776bc6b9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.879165 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347e480a70d549b08ece0b6958e885d4', 'request_id': 'req-13683e78-5dad-4b91-971a-b17b776bc6b9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.885306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347e480a70d549b08ece0b6958e885d4', 'request_id': 'req-13683e78-5dad-4b91-971a-b17b776bc6b9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.897325 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] The token cache is being invalidate because project 347e480a70d549b08ece0b6958e885d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:42:48.897619 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.898327 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:48.899893 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1429/2859] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:42:48 2022] DELETE /identity/v3/projects/347e480a70d549b08ece0b6958e885d4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:48.903903 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:48.914606 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd64e-1424-42a9-af08-5375b22c71fc service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:48.977659 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:48.978212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:48.978521 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:48.978790 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] PATH_INFO: `/v3/users/394a02c801a14ec499e9eb6fdb1eda94` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:48.979898 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=394a02c801a14ec499e9eb6fdb1eda94)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:48.981815 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:48.996127 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:49.200669 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394a02c801a14ec499e9eb6fdb1eda94'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:49.267249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394a02c801a14ec499e9eb6fdb1eda94'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:49.267868 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394a02c801a14ec499e9eb6fdb1eda94'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:49.280863 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394a02c801a14ec499e9eb6fdb1eda94'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:49.289325 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394a02c801a14ec499e9eb6fdb1eda94'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:49.298249 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1431/2860] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:42:48 2022] DELETE /identity/v3/users/394a02c801a14ec499e9eb6fdb1eda94 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:49.787358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:49.788816 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:49.802302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683e78-5dad-4b91-971a-b17b776bc6b9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:49.860917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:49.861553 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:49.861830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:49.862100 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] PATH_INFO: `/v3/projects/eea85c4a025d413aa87da248e6bfe11f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:49.863250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea85c4a025d413aa87da248e6bfe11f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:49.865190 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:50.004339 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea85c4a025d413aa87da248e6bfe11f', 'request_id': 'req-2b99a80e-9b32-4536-984a-6aa386f0046c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.077554 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea85c4a025d413aa87da248e6bfe11f', 'request_id': 'req-2b99a80e-9b32-4536-984a-6aa386f0046c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.078856 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:50.083807 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea85c4a025d413aa87da248e6bfe11f', 'request_id': 'req-2b99a80e-9b32-4536-984a-6aa386f0046c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.095460 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] The token cache is being invalidate because project eea85c4a025d413aa87da248e6bfe11f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:42:50.095749 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.096382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b99a80e-9b32-4536-984a-6aa386f0046c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.097861 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1430/2861] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:42:49 2022] DELETE /identity/v3/projects/eea85c4a025d413aa87da248e6bfe11f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:50.102011 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:50.103573 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:50.117141 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2fac-1803-4f1a-b6af-230e35eecfbd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:50.174781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:50.175409 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:50.175693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:50.175964 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] PATH_INFO: `/v3/users/72a56c5d22494235baa09ff4638d69b8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:50.177073 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a56c5d22494235baa09ff4638d69b8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:50.179128 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:50.367286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a56c5d22494235baa09ff4638d69b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.477179 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a56c5d22494235baa09ff4638d69b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.477806 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a56c5d22494235baa09ff4638d69b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.490389 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a56c5d22494235baa09ff4638d69b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.497031 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a56c5d22494235baa09ff4638d69b8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:50.503463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1432/2862] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:42:50 2022] DELETE /identity/v3/users/72a56c5d22494235baa09ff4638d69b8 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:50.649158 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb99a2-7d7d-4631-8396-7475e521ff5c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:50.649445 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb99a2-7d7d-4631-8396-7475e521ff5c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:50.649715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb99a2-7d7d-4631-8396-7475e521ff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:50.671915 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-61eb99a2-7d7d-4631-8396-7475e521ff5c None None] MFA Rules not processed for user `df0ffa9f450642cc94ed828cb34034b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:50.673123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-61eb99a2-7d7d-4631-8396-7475e521ff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:50.729905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1431/2863] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:50.956165 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:50.957755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:50.992257 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:51.049209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bacf7-8ec8-43b1-baec-68bd5bb208ba admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.054059 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.054749 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.055033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.055306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] PATH_INFO: `/v3/projects/01177607dede4eacb4319e1c761232d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.056456 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01177607dede4eacb4319e1c761232d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.058509 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.184020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01177607dede4eacb4319e1c761232d4', 'request_id': 'req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:51.271817 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01177607dede4eacb4319e1c761232d4', 'request_id': 'req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:51.278245 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01177607dede4eacb4319e1c761232d4', 'request_id': 'req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:51.291215 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] The token cache is being invalidate because project 01177607dede4eacb4319e1c761232d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:42:51.291507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:51.292148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-76a518a7-92c7-4b2a-94fa-c3225c6a95b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:42:51.293636 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1433/2864] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:42:50 2022] DELETE /identity/v3/projects/01177607dede4eacb4319e1c761232d4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:51.300136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.300636 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.300756 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.323507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:51.324781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:51.389594 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1432/2865] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:42:51.397626 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.397918 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.398195 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.419361 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:51.420855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:51.428869 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1434/2866] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:42:51.436597 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:51.447451 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f5e0a-6ae8-4637-9968-4e4c92ee8f1a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.452823 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.453425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.453705 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.453971 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.462842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:42:51.462842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.462842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.471735 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1433/2867] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:42:51.473106 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:51.482849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c72a8-6e3d-4225-b05a-07b87aa5e49d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.487750 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.488383 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.488665 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.488957 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.489645 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.491615 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.592211 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1435/2868] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:42:51 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:51.593425 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:51.603611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1e9d-6444-4182-9403-9d58a4d993d7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.672223 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.672854 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.673131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.673399 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.674043 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.676448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.780920 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1434/2869] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:42:51 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:42:51.785507 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:51.795628 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c494df9-84e3-4f34-abd1-6b12616f0c68 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.800256 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.813494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.813494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.813494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.813494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.813494 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.816735 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1436/2870] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:51.818035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:51.829034 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8587895-1827-44d6-82b2-451e736fef42 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.833712 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a15a5-35ea-4035-b874-60bb5ebedd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.834311 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a15a5-35ea-4035-b874-60bb5ebedd11 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.834637 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a15a5-35ea-4035-b874-60bb5ebedd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.834911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a15a5-35ea-4035-b874-60bb5ebedd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.835549 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a15a5-35ea-4035-b874-60bb5ebedd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.837143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a15a5-35ea-4035-b874-60bb5ebedd11 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.838548 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:51.849138 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1435/2871] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:42:51.853343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:51.863763 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189ff11-c590-4776-9c10-e3acf9982f35 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:51.868948 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:51.869607 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.869891 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.870194 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] PATH_INFO: `/v3/projects/53d6048fb19e4828a58b35024ced4518/users/7f3ca49df57440c0895389872f144036/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:51.885955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53d6048fb19e4828a58b35024ced4518, user_id=7f3ca49df57440c0895389872f144036, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:51.889320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:51.965261 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1437/2872] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:42:51 2022] PUT /identity/v3/projects/53d6048fb19e4828a58b35024ced4518/users/7f3ca49df57440c0895389872f144036/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:42:51.977198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89715e-6673-4a59-8145-f7c8e6d889d8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:51.977480 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89715e-6673-4a59-8145-f7c8e6d889d8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:51.977759 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89715e-6673-4a59-8145-f7c8e6d889d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:52.011168 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-df89715e-6673-4a59-8145-f7c8e6d889d8 None None] MFA Rules not processed for user `7f3ca49df57440c0895389872f144036`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:52.014288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-df89715e-6673-4a59-8145-f7c8e6d889d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:52.078030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1436/2873] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:42:52.089872 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:52.092583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:52.129955 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:42:52.160184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f221-51fb-4ab7-94a2-d616896313c4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:52.164337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'edb970ecb499474f87e02b9ee49c8c8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:52.165004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:52.165653 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:52.165914 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:52.177688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:52.179472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:52.191109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1438/2874] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:54.055815 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:54.056294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:54.056391 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:54.075274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] MFA Rules not processed for user `7f3ca49df57440c0895389872f144036`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:42:54.076970 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:54.083748 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1437/2875] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:42:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 03 01:42:54.120030 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:54.121462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:54.179163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688a2d6-68f0-43ed-9a59-6d641bbcd1d4 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:54.182736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc9f7e49e5946f48df125d4efd80dfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:54.183343 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:54.183627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:54.183899 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:54.186229 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:54.257889 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:54.259382 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844f3fc-5c87-4ba4-9acf-f911e61a4366 service glance] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:54.271271 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1439/2876] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:42:55.458741 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:42:55.460232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:55.526523 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075540c3-9ee6-41f5-ab78-2fabaf281e0e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:42:55.530830 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:42:55.531519 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:42:55.531813 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:42:55.532087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:42:55.533927 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:42:55.549369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:42:55.550860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:42:55.563526 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1438/2877] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:42:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:43:04.266252 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=67 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:05.557332 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=63 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:13.364747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccbe0a-d4c6-46d0-b11b-bf2d3bf2e625 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:13.365270 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccbe0a-d4c6-46d0-b11b-bf2d3bf2e625 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:13.365453 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccbe0a-d4c6-46d0-b11b-bf2d3bf2e625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:13.396640 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-efccbe0a-d4c6-46d0-b11b-bf2d3bf2e625 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:13.397793 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-efccbe0a-d4c6-46d0-b11b-bf2d3bf2e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:13.454979 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1440/2878] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:43:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:13.604358 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:13.605772 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:13.675232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a552fa-5e89-4973-afd3-5ba7a90a6bfb service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:13.680143 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:13.680746 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:13.681028 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:13.681301 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:13.689857 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:13.691658 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:13.699609 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1439/2879] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:43:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:43:14.521296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecd108-10b7-4b75-9451-7d15b2d5bcd9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:14.521746 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecd108-10b7-4b75-9451-7d15b2d5bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:14.521867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecd108-10b7-4b75-9451-7d15b2d5bcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:14.542503 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c4ecd108-10b7-4b75-9451-7d15b2d5bcd9 None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:14.543704 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c4ecd108-10b7-4b75-9451-7d15b2d5bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:14.551544 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1441/2880] 10.222.0.13 () {58 vars in 967 bytes} [Sat Sep 3 01:43:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:14.563525 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:14.574064 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d454d-7b4f-4137-b462-1e50995ca952 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:14.579142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:14.579734 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:14.580015 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:14.580288 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:14.588137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:14.589708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:14.598300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1440/2881] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:43:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:43:15.234417 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ecc17-356a-4b39-965e-c6cd1604bedc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:15.234707 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ecc17-356a-4b39-965e-c6cd1604bedc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:15.235004 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ecc17-356a-4b39-965e-c6cd1604bedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:15.254154 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ae7ecc17-356a-4b39-965e-c6cd1604bedc None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:15.255365 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ae7ecc17-356a-4b39-965e-c6cd1604bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:15.262328 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1442/2882] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Sep 3 01:43:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:15.274618 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:15.285047 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635d38d-605f-4f86-b6bc-c9d7ff3a7ffb service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:15.290179 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:15.290800 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:15.291078 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:15.291344 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:15.299236 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:15.300824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:15.308644 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1441/2883] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:43:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:43:15.943693 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:15.944161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:15.944263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:15.963624 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] MFA Rules not processed for user `57efcdf1033744358788130e1c663538`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:15.964747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:15.972296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1443/2884] 10.222.0.13 () {58 vars in 967 bytes} [Sat Sep 3 01:43:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:15.984086 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:15.994906 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8a535-150b-4dd0-bd92-baf776c6e6b3 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:15.999858 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a4455-0f03-4116-83bc-83eddd8ac1a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0711b98a2c214b7cb807f2d211b628c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:16.001014 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4455-0f03-4116-83bc-83eddd8ac1a6 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:16.001300 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4455-0f03-4116-83bc-83eddd8ac1a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:16.001571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a4455-0f03-4116-83bc-83eddd8ac1a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:16.009507 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a4455-0f03-4116-83bc-83eddd8ac1a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:16.011136 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a4455-0f03-4116-83bc-83eddd8ac1a6 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:16.019228 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1442/2885] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Sep 3 01:43:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 03 01:43:18.650148 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:18.712381 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cbbf8-5e2e-40d2-bf47-109f669f4eff None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:18.716826 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:18.717492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:18.717718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:18.717989 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] PATH_INFO: `/v3/users/7f3ca49df57440c0895389872f144036` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:18.719137 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3ca49df57440c0895389872f144036)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:18.721182 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:18.960222 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3ca49df57440c0895389872f144036'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.021015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3ca49df57440c0895389872f144036'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.021644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3ca49df57440c0895389872f144036'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.034757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3ca49df57440c0895389872f144036'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.040198 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:19.044936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3ca49df57440c0895389872f144036'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.047949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1444/2886] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:43:18 2022] DELETE /identity/v3/users/7f3ca49df57440c0895389872f144036 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:43:19.052981 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:19.053475 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:19.053599 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:19.072865 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:19.074062 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:19.132926 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1443/2887] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:19.143115 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:19.144553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:19.201084 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92364a75-38c4-4fdc-947b-d0caaebfb318 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:19.205161 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ce6eb-61ac-43a0-9aee-97bb45a39f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:19.205833 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ce6eb-61ac-43a0-9aee-97bb45a39f85 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:19.206112 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ce6eb-61ac-43a0-9aee-97bb45a39f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:19.206404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ce6eb-61ac-43a0-9aee-97bb45a39f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:19.215347 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ce6eb-61ac-43a0-9aee-97bb45a39f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:19.216779 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ce6eb-61ac-43a0-9aee-97bb45a39f85 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:19.226745 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1445/2888] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:43:19.577520 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:19.578999 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:19.594072 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968de2c1-2717-4120-83b7-702e53543a49 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:19.599013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:19.599652 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:19.599930 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:19.600201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] PATH_INFO: `/v3/projects/53d6048fb19e4828a58b35024ced4518` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:19.602123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53d6048fb19e4828a58b35024ced4518)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:19.604079 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:19.722501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:19.725247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53d6048fb19e4828a58b35024ced4518', 'request_id': 'req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.798818 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53d6048fb19e4828a58b35024ced4518', 'request_id': 'req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.805444 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53d6048fb19e4828a58b35024ced4518', 'request_id': 'req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.818026 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] The token cache is being invalidate because project 53d6048fb19e4828a58b35024ced4518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:43:19.818822 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.818982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c3b7a6be-e1b6-4c2f-ab35-23572e41e7b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:43:19.820494 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1444/2889] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:43:19 2022] DELETE /identity/v3/projects/53d6048fb19e4828a58b35024ced4518 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:43:19.825976 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:19.826523 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:19.826638 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:19.847478 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:19.848062 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:19.909158 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1446/2890] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:19.918003 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:19.918330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:19.918654 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:19.939374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:19.941241 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:19.949103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1445/2891] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:19.953334 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:19.962866 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5aef7-8f04-45a9-add5-9790a233e679 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:19.967485 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:19.968079 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:19.968379 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:19.968648 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:19.969286 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 03 01:43:19.969584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:19.971427 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:19.982768 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1447/2892] 10.222.0.13 () {58 vars in 1153 bytes} [Sat Sep 3 01:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 03 01:43:19.987921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:19.997336 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f75d7-5c17-45ea-8fed-fc5f9e680a82 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:20.001941 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:20.003131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.003458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.003670 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.004297 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:20.006184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:20.108458 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1446/2893] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:43:19 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:43:20.113369 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:20.124065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb29ff-7e29-4a12-8f58-31b986897f51 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:20.187564 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:20.188163 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.188448 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.188718 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.189335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:20.192103 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:20.272988 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1448/2894] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:43:20 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 03 01:43:20.277186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:20.288230 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5702e9-e537-4fee-a6fd-e79c691d3d8f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:20.292781 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:20.293375 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.293653 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.293921 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.294611 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:20.296198 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:20.307166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1447/2895] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:43:20.311081 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:20.325401 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450b753-cebd-433f-abd4-c39c9c8be089 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:20.325698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4138b-c63b-44f5-b477-e35bae673061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:20.326285 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4138b-c63b-44f5-b477-e35bae673061 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.326592 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4138b-c63b-44f5-b477-e35bae673061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.326861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4138b-c63b-44f5-b477-e35bae673061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.327505 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4138b-c63b-44f5-b477-e35bae673061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:20.329074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4138b-c63b-44f5-b477-e35bae673061 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:20.339617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1449/2896] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:43:20.343559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:20.353923 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645c8f4-f1a4-4727-9c81-04d64752d187 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:20.359824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:20.359824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.359824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.359824 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] PATH_INFO: `/v3/projects/4c46ebe5d6a74509ba46325e2376d170/users/7ebb858b78464e10a0fb011df133aaf1/roles/5d8beb80ba7c4b5a9147c7b3d12a6418` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.374588 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c46ebe5d6a74509ba46325e2376d170, user_id=7ebb858b78464e10a0fb011df133aaf1, role_id=5d8beb80ba7c4b5a9147c7b3d12a6418)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:20.377252 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:20.490201 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1448/2897] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:43:20 2022] PUT /identity/v3/projects/4c46ebe5d6a74509ba46325e2376d170/users/7ebb858b78464e10a0fb011df133aaf1/roles/5d8beb80ba7c4b5a9147c7b3d12a6418 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:43:20.495156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4f57b-85e2-4ad1-b1ae-1d15a4e112f1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.495443 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4f57b-85e2-4ad1-b1ae-1d15a4e112f1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.495717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4f57b-85e2-4ad1-b1ae-1d15a4e112f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.503855 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:20.514513 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3ff4f57b-85e2-4ad1-b1ae-1d15a4e112f1 None None] MFA Rules not processed for user `3188ed8aefba4edeb5f28b0b4a901aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:20.515706 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3ff4f57b-85e2-4ad1-b1ae-1d15a4e112f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:20.582781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1450/2898] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:20.592254 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:20.593960 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:20.651988 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2788e-beef-437c-a938-83a78b0137fb admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:20.655498 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:20.656104 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:20.656382 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:20.656651 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:20.666319 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:20.667766 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:20.677250 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1449/2899] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:43:26.242437 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:26.242963 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:26.243170 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:26.275617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] MFA Rules not processed for user `7ebb858b78464e10a0fb011df133aaf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:26.276843 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:26.345701 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1451/2900] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:26.350849 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:26.360905 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b072959-0d99-4aeb-b799-585b8d181c7f service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:26.366997 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:26.367197 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:26.367448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:26.367765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:26.368410 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:26.370320 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:26.497903 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1450/2901] 10.222.0.13 () {60 vars in 1153 bytes} [Sat Sep 3 01:43:26 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:43:26.501928 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:26.513884 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c51d1-19a4-497e-a7e7-2a90a8ef0cec None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:26.573100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:26.573717 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:26.574124 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:26.574429 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:26.575093 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:26.577028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:26.695599 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1452/2902] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Sep 3 01:43:26 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 03 01:43:26.699533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:26.714917 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a94ef9-11cc-4102-80b1-b41e9adcc238 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:26.715867 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:26.716559 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:26.716843 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:26.717119 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:26.717765 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:26.719437 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:26.733067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1451/2903] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:43:26.740109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:26.751477 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47ac8a-67c6-47ef-a29d-b2430b10e643 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:26.757212 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:26.757949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:26.758303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:26.758698 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] PATH_INFO: `/v3/projects/7ac9d587c54c4cdb8b2511639ce87d9c/users/adae8950471c4f3296854aa5fa8b0982/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:26.775463 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac9d587c54c4cdb8b2511639ce87d9c, user_id=adae8950471c4f3296854aa5fa8b0982, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:26.778263 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:26.886434 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:26.892005 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1453/2904] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:43:26 2022] PUT /identity/v3/projects/7ac9d587c54c4cdb8b2511639ce87d9c/users/adae8950471c4f3296854aa5fa8b0982/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:43:26.896294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:26.906513 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd1a04-2ca8-4e01-a82b-89ac689e4e35 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:26.949595 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:26.968987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71b478-6f8d-45e4-914e-5f7b67db7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:26.969607 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b478-6f8d-45e4-914e-5f7b67db7ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:26.969890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b478-6f8d-45e4-914e-5f7b67db7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:26.970163 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71b478-6f8d-45e4-914e-5f7b67db7ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:26.970850 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71b478-6f8d-45e4-914e-5f7b67db7ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:26.973306 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71b478-6f8d-45e4-914e-5f7b67db7ee9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:26.984792 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1452/2905] 10.222.0.13 () {58 vars in 1122 bytes} [Sat Sep 3 01:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 03 01:43:26.989265 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:26.999998 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69eeb8-b24a-4298-9602-c311af685fa4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:27.014740 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:27.015492 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:27.015774 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:27.016050 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] PATH_INFO: `/v3/projects/7ac9d587c54c4cdb8b2511639ce87d9c/users/adae8950471c4f3296854aa5fa8b0982/roles/00019edb8c5340468b5db8dc2c56fd73` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:27.018186 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac9d587c54c4cdb8b2511639ce87d9c, user_id=adae8950471c4f3296854aa5fa8b0982, role_id=00019edb8c5340468b5db8dc2c56fd73)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:27.021253 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:27.032980 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1454/2906] 10.222.0.13 () {60 vars in 1483 bytes} [Sat Sep 3 01:43:26 2022] PUT /identity/v3/projects/7ac9d587c54c4cdb8b2511639ce87d9c/users/adae8950471c4f3296854aa5fa8b0982/roles/00019edb8c5340468b5db8dc2c56fd73 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:43:32.806142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a762-0c8e-443b-85f7-d73b82d6c236 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:32.806142 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a762-0c8e-443b-85f7-d73b82d6c236 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:32.806827 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a762-0c8e-443b-85f7-d73b82d6c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:32.843087 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-38e0a762-0c8e-443b-85f7-d73b82d6c236 None None] MFA Rules not processed for user `adae8950471c4f3296854aa5fa8b0982`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:32.844419 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-38e0a762-0c8e-443b-85f7-d73b82d6c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:32.926946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1453/2907] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:32.944335 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:32.946227 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:33.033003 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd1d94-64de-45c7-8807-fe2d58fdd1a6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:33.040162 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:33.040787 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:33.041065 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:33.041515 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:33.108867 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:33.110472 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:33.121584 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1455/2908] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:43:36.385470 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:36.386372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:36.386372 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:36.406374 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] MFA Rules not processed for user `7ebb858b78464e10a0fb011df133aaf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:43:36.407603 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:36.415533 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1454/2909] 10.222.0.13 () {58 vars in 958 bytes} [Sat Sep 3 01:43:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:43:43.117378 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:46.401027 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=48 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:43:53.483311 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:53.484744 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:53.542747 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5ee47-3120-4db2-8811-883319bae7a3 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:53.546301 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:53.546927 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:53.547208 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:53.547480 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:53.549175 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:53.558629 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:53.560074 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:53.567949 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1456/2910] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:43:54.419631 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:54.421123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:54.492070 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda3cde-c711-423f-bee7-305f927eb74b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:54.497522 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:54.498131 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:54.498454 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:54.498729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:54.500448 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:54.512449 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:54.514063 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:54.523946 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1455/2911] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:43:58.078087 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:58.087298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108d4b2-2bbd-469f-a37d-65df5aaa9a14 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:58.091804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:58.092411 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:58.092692 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:58.092966 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:58.094797 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:58.105406 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:58.106895 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:58.126100 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1457/2912] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:43:58.878082 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:43:58.879598 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:58.896147 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9efcb-6925-4b00-bbf3-27b598f6655f service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:43:58.902383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:43:58.903035 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:43:58.903294 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:43:58.903555 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:43:58.905353 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:43:58.918639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:43:58.920281 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:43:58.932369 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1456/2913] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:43:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:44:08.112800 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=32 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:44:08.926624 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=38 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:44:11.058051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:44:11.070864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254c64-df1a-49f3-828e-a0306387febc service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:44:11.076352 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:44:11.077028 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:44:11.077306 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:44:11.077576 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:44:11.079305 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:44:11.091278 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:44:11.092849 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:44:11.103804 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1458/2914] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:44:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:44:13.276279 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:44:13.285911 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b186a-5a6d-4d42-ac21-c853864bc5aa service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:44:13.289878 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:44:13.291013 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:44:13.291296 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:44:13.291590 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:44:13.300290 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:44:13.301754 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:44:13.311883 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1457/2915] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:44:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:44:21.098695 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:44:23.307784 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:44:44.981770 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:44:44.994315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5e6b-b1a5-4a9c-a738-690046d95889 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:44:45.000680 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1749220d9e9547eba7ea035508c365bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:44:45.001303 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:44:45.001586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:44:45.001861 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:44:45.004320 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:44:45.018296 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:44:45.021136 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:44:45.033882 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1459/2916] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:44:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:44:55.028981 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:45:00.752738 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:45:00.762091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4eb5-abe2-43dc-a8c9-543786652760 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:45:00.765660 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:45:00.766251 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:45:00.766571 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:45:00.766842 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:45:00.768561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:45:00.831742 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:45:00.833123 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:45:00.843924 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1458/2917] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:45:07.963023 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:45:07.974541 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001e77a-69a8-42cd-8fda-321bc6b04b00 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:45:07.980778 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:45:07.981390 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:45:07.981676 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:45:07.981950 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:45:07.983732 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:45:07.996590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:45:07.998203 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:45:08.009302 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1460/2918] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:45:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:45:08.122780 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:45:08.124244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:45:08.137244 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da8de-29e0-4b3f-adc2-6e4bffe2683b service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:45:08.141896 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:45:08.142566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:45:08.142848 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:45:08.143116 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] PATH_INFO: `/v3/users/df0ffa9f450642cc94ed828cb34034b5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:45:08.144258 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0ffa9f450642cc94ed828cb34034b5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:45:08.146166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:45:08.313177 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0ffa9f450642cc94ed828cb34034b5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.313825 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0ffa9f450642cc94ed828cb34034b5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.326517 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0ffa9f450642cc94ed828cb34034b5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.333330 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0ffa9f450642cc94ed828cb34034b5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.414017 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0ffa9f450642cc94ed828cb34034b5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.420760 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1459/2919] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:45:08 2022] DELETE /identity/v3/users/df0ffa9f450642cc94ed828cb34034b5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:45:08.656075 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:45:08.669171 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fbd2e-9880-4434-8412-983cd1f377b9 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:45:08.727661 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:45:08.728297 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:45:08.728583 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:45:08.728864 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] PATH_INFO: `/v3/projects/c306d09e93584320b34697dba7641ac8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:45:08.729983 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c306d09e93584320b34697dba7641ac8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:45:08.732015 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:45:08.839590 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c306d09e93584320b34697dba7641ac8', 'request_id': 'req-eddad617-c0b2-4910-a0bf-a9fac6853cf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.888538 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c306d09e93584320b34697dba7641ac8', 'request_id': 'req-eddad617-c0b2-4910-a0bf-a9fac6853cf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.894890 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c306d09e93584320b34697dba7641ac8', 'request_id': 'req-eddad617-c0b2-4910-a0bf-a9fac6853cf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.906937 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] The token cache is being invalidate because project c306d09e93584320b34697dba7641ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:45:08.907169 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.907836 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:08.909337 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1461/2920] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:45:08 2022] DELETE /identity/v3/projects/c306d09e93584320b34697dba7641ac8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:45:08.913645 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:45:08.915105 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:45:08.928331 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc8c74-d712-4393-b924-cca04e796a0d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:45:08.987387 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:45:08.988022 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:45:08.988322 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:45:08.988587 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] PATH_INFO: `/v3/users/269c8434b1c64bd38729ac25ac1eaf48` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:45:08.989708 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] RBAC: Authorizing `identity:delete_user(user_id=269c8434b1c64bd38729ac25ac1eaf48)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:45:08.991922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:45:08.997581 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:45:09.186667 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269c8434b1c64bd38729ac25ac1eaf48'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.187302 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269c8434b1c64bd38729ac25ac1eaf48'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.199904 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269c8434b1c64bd38729ac25ac1eaf48'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.207195 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269c8434b1c64bd38729ac25ac1eaf48'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.265272 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269c8434b1c64bd38729ac25ac1eaf48'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.271557 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1460/2921] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:45:08 2022] DELETE /identity/v3/users/269c8434b1c64bd38729ac25ac1eaf48 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 03 01:45:09.523051 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:45:09.526753 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:45:09.541156 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddad617-c0b2-4910-a0bf-a9fac6853cf1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:45:09.600810 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:45:09.601462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:45:09.601743 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:45:09.602014 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] PATH_INFO: `/v3/projects/7bf08f2124de40729635b1c5d31b1dcd` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:45:09.603209 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf08f2124de40729635b1c5d31b1dcd)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:45:09.605142 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:45:09.627622 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:45:09.706762 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf08f2124de40729635b1c5d31b1dcd', 'request_id': 'req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.783283 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf08f2124de40729635b1c5d31b1dcd', 'request_id': 'req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.790754 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf08f2124de40729635b1c5d31b1dcd', 'request_id': 'req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.805644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] The token cache is being invalidate because project 7bf08f2124de40729635b1c5d31b1dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:45:09.805755 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.806444 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:45:09.807954 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1462/2922] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:45:09 2022] DELETE /identity/v3/projects/7bf08f2124de40729635b1c5d31b1dcd => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:45:19.267977 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:45:19.801481 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:46:12.571682 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:46:12.573643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:46:12.644420 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dcb7e2-9aba-47f2-919d-6c189803214c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:46:12.707966 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:46:12.708579 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:12.708862 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:12.709137 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:12.710980 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:46:12.723030 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:46:12.724617 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4b59e-9934-467f-8bf6-05c069772d12 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:46:12.735403 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1461/2923] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:46:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:46:22.731166 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=58 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:46:27.460319 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:46:27.462024 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:46:27.519781 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ca209-a2fb-4dd2-b05f-e077e8d2cc6f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:46:27.523371 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:46:27.523974 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:27.524284 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:27.524553 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:27.526272 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:46:27.535736 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:46:27.537462 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a7ea0-fe6c-4e92-a623-d1e77d2de69b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:46:27.545939 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1463/2924] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:46:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:46:28.142067 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c57bb9-bf81-4009-9f8a-71e0e938c6fb None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:28.142639 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c57bb9-bf81-4009-9f8a-71e0e938c6fb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:28.142870 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c57bb9-bf81-4009-9f8a-71e0e938c6fb None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:28.144379 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1462/2925] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:46:28.151038 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc316f6d-e544-40e8-9be1-941e13f36472 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:28.153404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc316f6d-e544-40e8-9be1-941e13f36472 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:28.153404 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc316f6d-e544-40e8-9be1-941e13f36472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:28.191008 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-bc316f6d-e544-40e8-9be1-941e13f36472 None None] MFA Rules not processed for user `d50713b3e4864430a69d0e00068257de`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:46:28.192234 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bc316f6d-e544-40e8-9be1-941e13f36472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:46:28.199581 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1464/2926] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:46:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:46:28.211103 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c57bb9-bf81-4009-9f8a-71e0e938c6fb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:46:28.222729 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c57bb9-bf81-4009-9f8a-71e0e938c6fb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:46:28.226187 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7d4a-ae31-443d-a899-5f7205a4ea0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:46:28.226808 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d4a-ae31-443d-a899-5f7205a4ea0e service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:28.227090 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d4a-ae31-443d-a899-5f7205a4ea0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:28.227383 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7d4a-ae31-443d-a899-5f7205a4ea0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:28.234273 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d4a-ae31-443d-a899-5f7205a4ea0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:46:28.235715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba7d4a-ae31-443d-a899-5f7205a4ea0e service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:46:28.243497 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1463/2927] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:46:28.324619 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb718e-057d-4610-9fa1-b8953b8f627a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:28.324911 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb718e-057d-4610-9fa1-b8953b8f627a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:28.325189 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb718e-057d-4610-9fa1-b8953b8f627a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:28.326644 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1465/2928] 10.222.0.13 () {54 vars in 921 bytes} [Sat Sep 3 01:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 03 01:46:28.332151 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:28.332434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:28.332702 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:28.371646 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] MFA Rules not processed for user `1749220d9e9547eba7ea035508c365bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 03 01:46:28.372787 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:46:28.439283 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1464/2929] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Sep 3 01:46:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 03 01:46:28.450223 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb718e-057d-4610-9fa1-b8953b8f627a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:46:28.461019 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb718e-057d-4610-9fa1-b8953b8f627a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:46:28.465524 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:46:28.466110 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:46:28.466416 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:46:28.466688 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:46:28.476550 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:46:28.478021 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:46:28.488290 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1466/2930] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:46:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:46:38.429890 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=43 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:46:38.483803 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=41 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:47:58.783531 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:47:58.795129 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86552005-f9e8-433d-8e25-05fe884646d8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:47:58.799987 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:47:58.800788 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:47:58.801071 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:47:58.801401 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:47:58.803574 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:47:58.819091 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:47:58.820659 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:47:58.833956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1465/2931] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:47:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 03 01:48:02.424045 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:48:02.427300 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:02.437037 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae431447-4409-45d1-b18e-04384600b9e6 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:48:02.440598 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd50713b3e4864430a69d0e00068257de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:48:02.441184 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:48:02.441459 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:48:02.441723 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:48:02.443552 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:02.458298 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:48:02.459757 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:48:02.471586 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1467/2932] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:48:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:48:08.827882 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:48:10.993503 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:48:10.995516 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:11.009806 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d78b-eaaf-4b9f-bd23-f207fbfa8bdd service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:48:11.015715 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:48:11.016313 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:48:11.016589 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:48:11.016855 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:48:11.018643 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:11.030169 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:48:11.031771 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:48:11.042232 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1466/2933] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:48:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:48:11.115627 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:48:11.117088 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:11.182612 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f99817-f3bb-4ff5-8d7b-1527193e4bb1 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:48:11.187033 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:48:11.187656 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:48:11.187935 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:48:11.188206 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] PATH_INFO: `/v3/users/7ebb858b78464e10a0fb011df133aaf1` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:48:11.189327 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ebb858b78464e10a0fb011df133aaf1)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:48:11.191293 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:48:11.336183 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ebb858b78464e10a0fb011df133aaf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:11.422879 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ebb858b78464e10a0fb011df133aaf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:11.423438 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ebb858b78464e10a0fb011df133aaf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:11.436495 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ebb858b78464e10a0fb011df133aaf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:11.443315 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ebb858b78464e10a0fb011df133aaf1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:11.449454 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1468/2934] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:48:11 2022] DELETE /identity/v3/users/7ebb858b78464e10a0fb011df133aaf1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:48:11.713934 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:48:11.726386 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba584c-110c-404e-a577-8a790b87a43c service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:48:11.784910 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:48:11.785544 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:48:11.785826 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:48:11.786097 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] PATH_INFO: `/v3/projects/4c46ebe5d6a74509ba46325e2376d170` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:48:11.787247 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c46ebe5d6a74509ba46325e2376d170)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:48:11.789417 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:48:11.912239 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c46ebe5d6a74509ba46325e2376d170', 'request_id': 'req-7d107080-e51f-4cee-a809-1db7c84dae24'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.002956 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c46ebe5d6a74509ba46325e2376d170', 'request_id': 'req-7d107080-e51f-4cee-a809-1db7c84dae24'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.009291 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c46ebe5d6a74509ba46325e2376d170', 'request_id': 'req-7d107080-e51f-4cee-a809-1db7c84dae24'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.021310 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] The token cache is being invalidate because project 4c46ebe5d6a74509ba46325e2376d170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:48:12.021608 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.022274 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.023796 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1467/2935] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:48:11 2022] DELETE /identity/v3/projects/4c46ebe5d6a74509ba46325e2376d170 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:48:12.027999 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:48:12.029617 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:12.043525 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58453416-1506-4380-a80f-5dc5177ec97c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:48:12.098785 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:48:12.099410 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:48:12.099691 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:48:12.099961 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] PATH_INFO: `/v3/users/adae8950471c4f3296854aa5fa8b0982` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:48:12.101080 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adae8950471c4f3296854aa5fa8b0982)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:48:12.103020 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:48:12.108930 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:48:12.279118 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adae8950471c4f3296854aa5fa8b0982'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.336880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adae8950471c4f3296854aa5fa8b0982'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.337497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adae8950471c4f3296854aa5fa8b0982'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.350218 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adae8950471c4f3296854aa5fa8b0982'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.357468 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adae8950471c4f3296854aa5fa8b0982'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.363880 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1469/2936] 10.222.0.13 () {58 vars in 1224 bytes} [Sat Sep 3 01:48:12 2022] DELETE /identity/v3/users/adae8950471c4f3296854aa5fa8b0982 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:48:12.598487 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:48:12.600058 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:48:12.615184 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d107080-e51f-4cee-a809-1db7c84dae24 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:48:12.675922 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3188ed8aefba4edeb5f28b0b4a901aed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06b321cae6a24e3a9217594a16dab365', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:48:12.676561 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:48:12.676838 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:48:12.677146 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] PATH_INFO: `/v3/projects/7ac9d587c54c4cdb8b2511639ce87d9c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:48:12.678256 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac9d587c54c4cdb8b2511639ce87d9c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:48:12.680186 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:48:12.701412 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:48:12.778434 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac9d587c54c4cdb8b2511639ce87d9c', 'request_id': 'req-6e548c0a-e936-42a0-a2db-b99f0ce6c060'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.868564 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac9d587c54c4cdb8b2511639ce87d9c', 'request_id': 'req-6e548c0a-e936-42a0-a2db-b99f0ce6c060'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.875431 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac9d587c54c4cdb8b2511639ce87d9c', 'request_id': 'req-6e548c0a-e936-42a0-a2db-b99f0ce6c060'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.886566 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] The token cache is being invalidate because project 7ac9d587c54c4cdb8b2511639ce87d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 03 01:48:12.886860 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.887501 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6e548c0a-e936-42a0-a2db-b99f0ce6c060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 03 01:48:12.888982 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1468/2937] 10.222.0.13 () {58 vars in 1233 bytes} [Sat Sep 3 01:48:12 2022] DELETE /identity/v3/projects/7ac9d587c54c4cdb8b2511639ce87d9c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 03 01:48:22.364845 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:48:22.883471 dv-cinder-838009-7-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 03 01:51:14.876497 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 03 01:51:14.877945 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:51:15.015936 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059f1ec-53a4-43d5-8f0f-28beff737b51 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 03 01:51:15.073470 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54b10e8ec2e244898f250cb987b6c7c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f9529286db449018f980b53c7eb4e67', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 03 01:51:15.074109 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 03 01:51:15.074422 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 03 01:51:15.074694 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 03 01:51:15.076419 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 03 01:51:15.087346 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 03 01:51:15.088922 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb423cb0-5a39-4259-af77-64ffe2f97084 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 03 01:51:15.098748 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1470/2938] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Sep 3 01:51:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 03 01:51:25.094616 dv-cinder-838009-7-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}